Degrees of Freedom (statistics) - Degrees of Freedom of A Random Vector

Degrees of Freedom of A Random Vector

Geometrically, the degrees of freedom can be interpreted as the dimension of certain vector subspaces. As a starting point, suppose that we have a sample of n independent normally distributed observations,

.

This can be represented as an n-dimensional random vector:

Since this random vector can lie anywhere in n-dimensional space, it has n degrees of freedom.

Now, let be the sample mean. The random vector can be decomposed as the sum of the sample mean plus a vector of residuals:

The first vector on the right-hand side is constrained to be a multiple of the vector of 1's, and the only free quantity is . It therefore has 1 degree of freedom.

The second vector is constrained by the relation . The first n − 1 components of this vector can be anything. However, once you know the first n − 1 components, the constraint tells you the value of the nth component. Therefore, this vector has n − 1 degrees of freedom.

Mathematically, the first vector is the orthogonal, or least-squares, projection of the data vector onto the subspace spanned by the vector of 1's. The 1 degree of freedom is the dimension of this subspace. The second residual vector is the least-squares projection onto the (n − 1)-dimensional orthogonal complement of this subspace, and has n − 1 degrees of freedom.

In statistical testing applications, often one isn't directly interested in the component vectors, but rather in their squared lengths. In the example above, the residual sum-of-squares is

If the data points are normally distributed with mean 0 and variance, then the residual sum of squares has a scaled chi-squared distribution (scaled by the factor ), with n − 1 degrees of freedom. The degrees-of-freedom, here a parameter of the distribution, can still be interpreted as the dimension of an underlying vector subspace.

Likewise, the one-sample t-test statistic,

follows a Student's t distribution with n − 1 degrees of freedom when the hypothesized mean is correct. Again, the degrees-of-freedom arises from the residual vector in the denominator.