Primal-dual Interior Point Method For Nonlinear Optimization
The primal-dual method's idea is easy to demonstrate for constrained nonlinear optimization. For simplicity consider the all-inequality version of a nonlinear optimization problem:
- minimize subject to .
The logarithmic barrier function associated with (1) is
Here is a small positive scalar, sometimes called the "barrier parameter". As converges to zero the minimum of should converge to a solution of (1).
The barrier function gradient is
where is the gradient of the original function and is the gradient of .
In addition to the original ("primal") variable we introduce a Lagrange multiplier inspired dual variable (sometimes called "slack variable")
(4) is sometimes called the "perturbed complementarity" condition, for its resemblance to "complementary slackness" in KKT conditions.
We try to find those which turn gradient of barrier function to zero.
Applying (4) to (3) we get equation for gradient:
where the matrix is the constraint Jacobian.
The intuition behind (5) is that the gradient of should lie in the subspace spanned by the constraints' gradients. The "perturbed complementarity" with small (4) can be understood as the condition that the solution should either lie near the boundary or that the projection of the gradient on the constraint component normal should be almost zero.
Applying Newton's method to (4) and (5) we get an equation for update :
where is the Hessian matrix of and is a diagonal matrix of .
Because of (1), (4) the condition
should be enforced at each step. This can be done by choosing appropriate :
- .
Read more about this topic: Interior Point Method
Famous quotes containing the words interior, point and/or method:
“Anyone with a real taste for solitude who indulges that taste encounters the dangers of any other drug-taker. The habit grows. You become an addict.... Absorbed in the visions of solitude, human beings are only interruptions. What voice can equal the voices of solitude? What sights equal the movement of a single day’s tide of light across the floor boards of one room? What drama be as continuously absorbing as the interior one?”
—Jessamyn West (1902–1984)
“All Coolidge had to do in 1924 was to keep his mean trap shut, to be elected. All Harding had to do in 1920 was repeat “Avoid foreign entanglements.” All Hoover had to do in 1928 was to endorse Coolidge. All Roosevelt had to do in 1932 was to point to Hoover.”
—Robert E. Sherwood (1896–1955)
“No method nor discipline can supersede the necessity of being forever on the alert. What is a course of history or philosophy, or poetry, no matter how well selected, or the best society, or the most admirable routine of life, compared with the discipline of looking always at what is to be seen? Will you be a reader, a student merely, or a seer? Read your fate, see what is before you, and walk on into futurity.”
—Henry David Thoreau (1817–1862)