In optimization, a descent direction is a vector that, in the sense below, moves us closer towards a local minimum of our objective function .
Suppose we are computing by an iterative method, such as line search. We define a descent direction at the th iterate to be any such that, where denotes the inner product. The motivation for such an approach is that small steps along guarantee that is reduced, by Taylor's theorem.
Using this definition, the negative of a non-zero gradient is always a descent direction, as .
Numerous methods exist to compute descent directions, all with differing merits. For example, one could use gradient descent or the conjugate gradient method.
Famous quotes containing the words descent and/or direction:
“And every stone shall cry,
In praises of the child
By whose descent among us
The worlds are reconciled.”
—Richard Wilbur (b. 1921)
“Each man has his own vocation. The talent is the call. There is one direction in which all space is open to him. He has faculties silently inviting him thither to endless exertion. He is like a ship in the river; he runs against obstructions on every side but one; on that side all obstruction is taken away, and he sweeps serenely over a deepening channel into an infinite sea.”
—Ralph Waldo Emerson (1803–1882)