Primal-dual Interior Point Method For Nonlinear Optimization
The primal-dual method's idea is easy to demonstrate for constrained nonlinear optimization. For simplicity consider the all-inequality version of a nonlinear optimization problem:
- minimize subject to .
The logarithmic barrier function associated with (1) is
Here is a small positive scalar, sometimes called the "barrier parameter". As converges to zero the minimum of should converge to a solution of (1).
The barrier function gradient is
where is the gradient of the original function and is the gradient of .
In addition to the original ("primal") variable we introduce a Lagrange multiplier inspired dual variable (sometimes called "slack variable")
(4) is sometimes called the "perturbed complementarity" condition, for its resemblance to "complementary slackness" in KKT conditions.
We try to find those which turn gradient of barrier function to zero.
Applying (4) to (3) we get equation for gradient:
where the matrix is the constraint Jacobian.
The intuition behind (5) is that the gradient of should lie in the subspace spanned by the constraints' gradients. The "perturbed complementarity" with small (4) can be understood as the condition that the solution should either lie near the boundary or that the projection of the gradient on the constraint component normal should be almost zero.
Applying Newton's method to (4) and (5) we get an equation for update :
where is the Hessian matrix of and is a diagonal matrix of .
Because of (1), (4) the condition
should be enforced at each step. This can be done by choosing appropriate :
- .
Read more about this topic: Interior Point Method
Famous quotes containing the words interior, point and/or method:
“Professor Eucalyptus said, “The search
For reality is as momentous as
The search for god.” It is the philosopher’s search
For an interior made exterior
And the poet’s search for the same exterior made
Interior: breathless things broodingly abreath....”
—Wallace Stevens (1879–1955)
“Film is more than the twentieth-century art. It’s another part of the twentieth-century mind. It’s the world seen from inside. We’ve come to a certain point in the history of film. If a thing can be filmed, the film is implied in the thing itself. This is where we are. The twentieth century is on film.... You have to ask yourself if there’s anything about us more important than the fact that we’re constantly on film, constantly watching ourselves.”
—Don Delillo (b. 1926)
“A method of child-rearing is not—or should not be—a whim, a fashion or a shibboleth. It should derive from an understanding of the developing child, of his physical and mental equipment at any given stage, and, therefore, his readiness at any given stage to adapt, to learn, to regulate his behavior according to parental expectations.”
—Selma H. Fraiberg (20th century)