Degrees of Freedom of A Random Vector
Geometrically, the degrees of freedom can be interpreted as the dimension of certain vector subspaces. As a starting point, suppose that we have a sample of n independent normally distributed observations,
- .
This can be represented as an n-dimensional random vector:
Since this random vector can lie anywhere in n-dimensional space, it has n degrees of freedom.
Now, let be the sample mean. The random vector can be decomposed as the sum of the sample mean plus a vector of residuals:
The first vector on the right-hand side is constrained to be a multiple of the vector of 1's, and the only free quantity is . It therefore has 1 degree of freedom.
The second vector is constrained by the relation . The first n − 1 components of this vector can be anything. However, once you know the first n − 1 components, the constraint tells you the value of the nth component. Therefore, this vector has n − 1 degrees of freedom.
Mathematically, the first vector is the orthogonal, or least-squares, projection of the data vector onto the subspace spanned by the vector of 1's. The 1 degree of freedom is the dimension of this subspace. The second residual vector is the least-squares projection onto the (n − 1)-dimensional orthogonal complement of this subspace, and has n − 1 degrees of freedom.
In statistical testing applications, often one isn't directly interested in the component vectors, but rather in their squared lengths. In the example above, the residual sum-of-squares is
If the data points are normally distributed with mean 0 and variance, then the residual sum of squares has a scaled chi-squared distribution (scaled by the factor ), with n − 1 degrees of freedom. The degrees-of-freedom, here a parameter of the distribution, can still be interpreted as the dimension of an underlying vector subspace.
Likewise, the one-sample t-test statistic,
follows a Student's t distribution with n − 1 degrees of freedom when the hypothesized mean is correct. Again, the degrees-of-freedom arises from the residual vector in the denominator.
Read more about this topic: Degrees Of Freedom (statistics)
Famous quotes containing the words degrees of, degrees, freedom and/or random:
“So that the life of a writer, whatever he might fancy to the contrary, was not so much a state of composition, as a state of warfare; and his probation in it, precisely that of any other man militant upon earth,—both depending alike, not half so much upon the degrees of his WIT—as his RESISTANCE.”
—Laurence Sterne (1713–1768)
“How have I been able to live so long outside Nature without identifying myself with it? Everything lives, moves, everything corresponds; the magnetic rays, emanating either from myself or from others, cross the limitless chain of created things unimpeded; it is a transparent network that covers the world, and its slender threads communicate themselves by degrees to the planets and stars. Captive now upon earth, I commune with the chorus of the stars who share in my joys and sorrows.”
—Gérard De Nerval (1808–1855)
“The reason artists show so little interest
In public freedom is because the freedom
They’ve come to feel the need of is a kind
No one can give them they can scarce attain
The freedom of their own material....”
—Robert Frost (1874–1963)
“We should stop looking to law to provide the final answer.... Law cannot save us from ourselves.... We have to go out and try to accomplish our goals and resolve disagreements by doing what we think is right. That energy and resourcefulness, not millions of legal cubicles, is what was great about America. Let judgment and personal conviction be important again.”
—Philip K. Howard, U.S. lawyer. The Death of Common Sense: How Law Is Suffocating America, pp. 186-87, Random House (1994)