Linear Least Squares (mathematics) - Constrained Linear Least Squares

Constrained Linear Least Squares

Often it is of interest to solve a linear least squares problem with an additional constraint on the solution. With constrained linear least squares, the original equation

must be satisfied (in the least squares sense) while also ensuring that some other property of is maintained. There are often special purpose algorithms for solving such problems efficiently. Some examples of constraints are given below:

• Equality constrained least squares: the elements of must exactly satisfy
• Regularized least squares: the elements of must satisfy
• Non-negative least squares (NNLS): The vector satisfies the vector inequality that is defined componentwise --- that is, each component must be either positive or zero.
• Box-constrained least squares: The vector satisfies the vector inequalities, each of which is defined componentwise.
• Integer constrained least squares: all elements of must be integer (instead of real numbers).
• Real constrained least squares: all elements of must be real (rather than complex numbers).
• Phase constrained least squares: all elements of must have the same phase.

When the constraint only applies to some of the variables, the mixed problem may be solved using separable least squares by letting and represent the unconstrained (1) and constrained (2) components. Then substituting the least squares solution for, i.e.

back into the original expression gives (following some rearrangement) an equation that can be solved as a purely constrained problem in .

where is a projection matrix. Following the constrained estimation of the vector is obtained from the expression above.