Lack-of-fit Sum of Squares - Mathematical Details

Mathematical Details

Consider fitting a line with one predictor variable. Define i as an index of each of the n distinct x values, j as an index of the response variable observations for a given x value, and ni as the number of y values associated with the i th x value. The value of each response variable observation can be represented by

Let

be the least squares estimates of the unobservable parameters α and β based on the observed values of x i and Y i j.

Let

be the fitted values of the response variable. Then

are the residuals, which are observable estimates of the unobservable values of the error term ε ij. Because of the nature of the method of least squares, the whole vector of residuals, with

scalar components, necessarily satisfies the two constraints

It is thus constrained to lie in an (N − 2)-dimensional subspace of R N, i.e. there are N − 2 "degrees of freedom for error".

Now let

be the average of all Y-values associated with the i th x-value.

We partition the sum of squares due to error into two components:

\begin{align} & \sum_{i=1}^n \sum_{j=1}^{n_i} \widehat\varepsilon_{ij}^{\,2} = \sum_{i=1}^n \sum_{j=1}^{n_i} \left( Y_{ij} - \widehat Y_i \right)^2 \\ & = \underbrace{ \sum_{i=1}^n \sum_{j=1}^{n_i} \left(Y_{ij} - \overline Y_{i\bullet}\right)^2 }_\text{(sum of squares due to pure error)} + \underbrace{ \sum_{i=1}^n n_i \left( \overline Y_{i\bullet} - \widehat Y_i \right)^2. }_\text{(sum of squares due to lack of fit)} \end{align}

Read more about this topic:  Lack-of-fit Sum Of Squares

Famous quotes containing the words mathematical and/or details:

    “All science requires mathematics. The knowledge of mathematical things is almost innate in us.... This is the easiest of sciences, a fact which is obvious in that no one’s brain rejects it; for laymen and people who are utterly illiterate know how to count and reckon.”
    —Roger Bacon (c. 1214–c. 1294)

    “There was a time when the average reader read a novel simply for the moral he could get out of it, and however naïve that may have been, it was a good deal less naïve than some of the limited objectives he has now. Today novels are considered to be entirely concerned with the social or economic or psychological forces that they will by necessity exhibit, or with those details of daily life that are for the good novelist only means to some deeper end.”
    —Flannery O’Connor (1925–1964)