Lagrange Multiplier - Introduction

Introduction

One of the most common problems in calculus is that of finding maxima or minima (in general, "extrema") of a function, but it is often difficult to find a closed form for the function being extremized. Such difficulties often arise when one wishes to maximize or minimize a function subject to fixed outside conditions or constraints. The method of Lagrange multipliers is a powerful tool for solving this class of problems without the need to explicitly solve the conditions and use them to eliminate extra variables.

Consider the two-dimensional problem introduced above:

maximize

subject to

We can visualize contours of f given by

for various values of, and the contour of given by .

Suppose we walk along the contour line with . In general the contour lines of and may be distinct, so following the contour line for one could intersect with or cross the contour lines of . This is equivalent to saying that while moving along the contour line for the value of can vary. Only when the contour line for meets contour lines of tangentially, do we not increase or decrease the value of — that is, when the contour lines touch but do not cross.

The contour lines of f and g touch when the tangent vectors of the contour lines are parallel. Since the gradient of a function is perpendicular to the contour lines, this is the same as saying that the gradients of f and g are parallel. Thus we want points where and

,

where

and

are the respective gradients. The constant is required because although the two gradient vectors are parallel, the magnitudes of the gradient vectors are generally not equal.

To incorporate these conditions into one equation, we introduce an auxiliary function

and solve

This is the method of Lagrange multipliers. Note that implies .