In optimization, a descent direction is a vector that, in the sense below, moves us closer towards a local minimum of our objective function .
Suppose we are computing by an iterative method, such as line search. We define a descent direction at the th iterate to be any such that, where denotes the inner product. The motivation for such an approach is that small steps along guarantee that is reduced, by Taylor's theorem.
Using this definition, the negative of a non-zero gradient is always a descent direction, as .
Numerous methods exist to compute descent directions, all with differing merits. For example, one could use gradient descent or the conjugate gradient method.
Famous quotes containing the words descent and/or direction:
““There is Hawthorne, with genius so shrinking and rare
That you hardly at first see the strength that is there;
A frame so robust, with a nature so sweet,
So earnest, so graceful, so lithe and so fleet,
Is worth a descent from Olympus to meet;”
—James Russell Lowell (1819–1891)
“Each man has his own vocation. The talent is the call. There is one direction in which all space is open to him. He has faculties silently inviting him thither to endless exertion. He is like a ship in the river; he runs against obstructions on every side but one; on that side all obstruction is taken away, and he sweeps serenely over a deepening channel into an infinite sea.”
—Ralph Waldo Emerson (1803–1882)