Linear Least Squares (mathematics) - The General Problem

The General Problem

Consider an overdetermined system

of m linear equations in n unknown coefficients, β1,β2,…,βn, with m > n. This can be written in matrix form as

where

\mathbf {X}=\begin{pmatrix} X_{11} & X_{12} & \cdots & X_{1n} \\ X_{21} & X_{22} & \cdots & X_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ X_{m1} & X_{m2} & \cdots & X_{mn} \end{pmatrix}, \qquad \boldsymbol \beta = \begin{pmatrix} \beta_1 \\ \beta_2 \\ \vdots \\ \beta_n \end{pmatrix}, \qquad \mathbf y = \begin{pmatrix} y_1 \\ y_2 \\ \vdots \\ y_m \end{pmatrix}.

Such a system usually has no solution, so the goal is instead to find the coefficients β which fit the equations "best," in the sense of solving the quadratic minimization problem

where the objective function S is given by

A justification for choosing this criterion is given in properties below. This minimization problem has a unique solution, provided that the n columns of the matrix X are linearly independent, given by solving the normal equations

Read more about this topic:  Linear Least Squares (mathematics)

Famous quotes containing the words general and/or problem:

    “The man who would change the name of Arkansas is the original, iron-jawed, brass-mouthed, copper-bellied corpse-maker from the wilds of the Ozarks! He is the man they call Sudden Death and General Desolation! Sired by a hurricane, dam’d by an earthquake, half-brother to the cholera, nearly related to the smallpox on his mother’s side!”
    —Administration in the State of Arka, U.S. public relief program (1935-1943)

    “Will women find themselves in the same position they have always been? Or do we see liberation as solving the conditions of women in our society?... If we continue to shy away from this problem we will not be able to solve it after independence. But if we can say that our first priority is the emancipation of women, we will become free as members of an oppressed community.”
    —Ruth Mompati (b. 1925)