Linear Least Squares (mathematics) - The General Problem

The General Problem

Consider an overdetermined system

of m linear equations in n unknown coefficients, β1,β2,…,βn, with m > n. This can be written in matrix form as

where

\mathbf {X}=\begin{pmatrix} X_{11} & X_{12} & \cdots & X_{1n} \\ X_{21} & X_{22} & \cdots & X_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ X_{m1} & X_{m2} & \cdots & X_{mn} \end{pmatrix}, \qquad \boldsymbol \beta = \begin{pmatrix} \beta_1 \\ \beta_2 \\ \vdots \\ \beta_n \end{pmatrix}, \qquad \mathbf y = \begin{pmatrix} y_1 \\ y_2 \\ \vdots \\ y_m \end{pmatrix}.

Such a system usually has no solution, so the goal is instead to find the coefficients β which fit the equations "best," in the sense of solving the quadratic minimization problem

where the objective function S is given by

A justification for choosing this criterion is given in properties below. This minimization problem has a unique solution, provided that the n columns of the matrix X are linearly independent, given by solving the normal equations

Read more about this topic:  Linear Least Squares (mathematics)

Famous quotes containing the words general and/or problem:

    “Must I remind you that a chain is no stronger than its weakest link?”
    —Jerome Cady, U.S. screenwriter, and Lewis Milestone. General Mitsubi (Richard Loo)

    “Congress seems drugged and inert most of the time. ...Its idea of meeting a problem is to hold hearings or, in extreme cases, to appoint a commission.”
    —Shirley Chisholm (b. 1924)