Optimal Control - Linear Quadratic Control

Linear Quadratic Control

A special case of the general nonlinear optimal control problem given in the previous section is the linear quadratic (LQ) optimal control problem. The LQ problem is stated as follows. Minimize the quadratic continuous-time cost functional

Subject to the linear first-order dynamic constraints

and the initial condition

A particular form of the LQ problem that arises in many control system problems is that of the linear quadratic regulator (LQR) where all of the matrices (i.e., and ) are constant, the initial time is arbitrarily set to zero, and the terminal time is taken in the limit (this last assumption is what is known as infinite horizon). The LQR problem is stated as follows. Minimize the infinite horizon quadratic continuous-time cost functional

Subject to the linear time-invariant first-order dynamic constraints

and the initial condition

In the finite-horizon case the matrices are restricted in that and are positive semi-definite and positive definite, respectively. In the infinite-horizon case, however, the matrices and are not only positive-semidefinite and positive-definite, respectively, but are also constant. These additional restrictions on and in the infinite-horizon case are enforced to ensure that the cost functional remains positive. Furthermore, in order to ensure that the cost function is bounded, the additional restriction is imposed that the pair is controllable. Note that the LQ or LQR cost functional can be thought of physically as attempting to minimize the control energy (measured as a quadratic form).

The infinite horizon problem (i.e., LQR) may seem overly restrictive and essentially useless because it assumes that the operator is driving the system to zero-state and hence driving the output of the system to zero. This is indeed correct. However the problem of driving the output to a desired nonzero level can be solved after the zero output one is. In fact, it can be proved that this secondary LQR problem can be solved in a very straightforward manner. It has been shown in classical optimal control theory that the LQ (or LQR) optimal control has the feedback form

where is a properly dimensioned matrix, given as

and is the solution of the differential Riccati equation. The differential Riccati equation is given as

For the finite horizon LQ problem, the Riccati equation is integrated backward in time using the terminal boundary condition

For the infinite horizon LQR problem, the differential Riccati equation is replaced with the algebraic Riccati equation (ARE) given as

Understanding that the ARE arises from infinite horizon problem, the matrices, and are all constant. It is noted that there are in general multiple solutions to the algebraic Riccati equation and the positive definite (or positive semi-definite) solution is the one that is used to compute the feedback gain. The LQ (LQR) problem was elegantly solved by Rudolf Kalman.