Linear Least Squares (mathematics) - The General Problem

The General Problem

Consider an overdetermined system

of m linear equations in n unknown coefficients, β1,β2,…,βn, with m > n. This can be written in matrix form as

where

\mathbf {X}=\begin{pmatrix} X_{11} & X_{12} & \cdots & X_{1n} \\ X_{21} & X_{22} & \cdots & X_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ X_{m1} & X_{m2} & \cdots & X_{mn} \end{pmatrix}, \qquad \boldsymbol \beta = \begin{pmatrix} \beta_1 \\ \beta_2 \\ \vdots \\ \beta_n \end{pmatrix}, \qquad \mathbf y = \begin{pmatrix} y_1 \\ y_2 \\ \vdots \\ y_m \end{pmatrix}.

Such a system usually has no solution, so the goal is instead to find the coefficients β which fit the equations "best," in the sense of solving the quadratic minimization problem

where the objective function S is given by

A justification for choosing this criterion is given in properties below. This minimization problem has a unique solution, provided that the n columns of the matrix X are linearly independent, given by solving the normal equations

Read more about this topic:  Linear Least Squares (mathematics)

Famous quotes containing the words general and/or problem:

    “The tremendous outflow of intellectuals that formed such a prominent part of the general exodus from Soviet Russia in the first years of the Bolshevist Revolution seems today like the wanderings of some mythical tribe whose bird-signs and moon-signs I now retrieve from the desert dust.”
    —Vladimir Nabokov (1899–1977)

    “Hypocrisy is the essence of snobbery, but all snobbery is about the problem of belonging.”
    —Alexander Theroux (b. 1940)