Linear Least Squares (mathematics) - The General Problem

The General Problem

Consider an overdetermined system

of m linear equations in n unknown coefficients, β1,β2,…,βn, with m > n. This can be written in matrix form as

where

\mathbf {X}=\begin{pmatrix} X_{11} & X_{12} & \cdots & X_{1n} \\ X_{21} & X_{22} & \cdots & X_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ X_{m1} & X_{m2} & \cdots & X_{mn} \end{pmatrix}, \qquad \boldsymbol \beta = \begin{pmatrix} \beta_1 \\ \beta_2 \\ \vdots \\ \beta_n \end{pmatrix}, \qquad \mathbf y = \begin{pmatrix} y_1 \\ y_2 \\ \vdots \\ y_m \end{pmatrix}.

Such a system usually has no solution, so the goal is instead to find the coefficients β which fit the equations "best," in the sense of solving the quadratic minimization problem

where the objective function S is given by

A justification for choosing this criterion is given in properties below. This minimization problem has a unique solution, provided that the n columns of the matrix X are linearly independent, given by solving the normal equations

Read more about this topic:  Linear Least Squares (mathematics)

Famous quotes containing the words general and/or problem:

    “No government can help the destinies of people who insist in putting sectional and class consciousness ahead of general weal.”
    —Franklin D. Roosevelt (1882–1945)

    “What happened at Hiroshima was not only that a scientific breakthrough ... had occurred and that a great part of the population of a city had been burned to death, but that the problem of the relation of the triumphs of modern science to the human purposes of man had been explicitly defined.”
    —Archibald MacLeish (1892–1982)