Primal-dual Interior Point Method For Nonlinear Optimization
The primal-dual method's idea is easy to demonstrate for constrained nonlinear optimization. For simplicity consider the all-inequality version of a nonlinear optimization problem:
- minimize subject to .
The logarithmic barrier function associated with (1) is
Here is a small positive scalar, sometimes called the "barrier parameter". As converges to zero the minimum of should converge to a solution of (1).
The barrier function gradient is
where is the gradient of the original function and is the gradient of .
In addition to the original ("primal") variable we introduce a Lagrange multiplier inspired dual variable (sometimes called "slack variable")
(4) is sometimes called the "perturbed complementarity" condition, for its resemblance to "complementary slackness" in KKT conditions.
We try to find those which turn gradient of barrier function to zero.
Applying (4) to (3) we get equation for gradient:
where the matrix is the constraint Jacobian.
The intuition behind (5) is that the gradient of should lie in the subspace spanned by the constraints' gradients. The "perturbed complementarity" with small (4) can be understood as the condition that the solution should either lie near the boundary or that the projection of the gradient on the constraint component normal should be almost zero.
Applying Newton's method to (4) and (5) we get an equation for update :
where is the Hessian matrix of and is a diagonal matrix of .
Because of (1), (4) the condition
should be enforced at each step. This can be done by choosing appropriate :
- .
Read more about this topic: Interior Point Method
Famous quotes containing the words interior, point and/or method:
“Or would interior time, that could delay
The sentence chronic with the last assize,
Start running backwards with its timely lies,
I might have time to live the love I say....”
—Allen Tate (1899–1979)
“The professional celebrity, male and female, is the crowning result of the star system of a society that makes a fetish of competition. In America, this system is carried to the point where a man who can knock a small white ball into a series of holes in the ground with more efficiency than anyone else thereby gains social access to the President of the United States.”
—C. Wright Mills (1916–1962)
“In child rearing it would unquestionably be easier if a child were to do something because we say so. The authoritarian method does expedite things, but it does not produce independent functioning. If a child has not mastered the underlying principles of human interactions and merely conforms out of coercion or conditioning, he has no tools to use, no resources to apply in the next situation that confronts him.”
—Elaine Heffner (20th century)