Degrees of Freedom in Linear Models
The demonstration of the t and chi-squared distributions for one-sample problems above is the simplest example where degrees-of-freedom arise. However, similar geometry and vector decompositions underlie much of the theory of linear models, including linear regression and analysis of variance. An explicit example based on comparison of three means is presented here; the geometry of linear models is discussed in more complete detail by Christensen (2002).
Suppose independent observations are made for three populations, and . The restriction to three groups and equal sample sizes simplifies notation, but the ideas are easily generalized.
The observations can be decomposed as
where are the means of the individual samples, and is the mean of all 3n observations. In vector notation this decomposition can be written as
The observation vector, on the left-hand side, has 3n degrees of freedom. On the right-hand side, the first vector has one degree of freedom (or dimension) for the overall mean. The second vector depends on three random variables, and . However, these must sum to 0 and so are constrained; the vector therefore must lie in a 2-dimensional subspace, and has 2 degrees of freedom. The remaining 3n − 3 degrees of freedom are in the residual vector (made up of n − 1 degrees of freedom within each of the populations).
Read more about this topic: Degrees Of Freedom (statistics)
Famous quotes containing the words degrees of, degrees, freedom and/or models:
“So that the life of a writer, whatever he might fancy to the contrary, was not so much a state of composition, as a state of warfare; and his probation in it, precisely that of any other man militant upon earth,—both depending alike, not half so much upon the degrees of his WIT—as his RESISTANCE.”
—Laurence Sterne (1713–1768)
“How have I been able to live so long outside Nature without identifying myself with it? Everything lives, moves, everything corresponds; the magnetic rays, emanating either from myself or from others, cross the limitless chain of created things unimpeded; it is a transparent network that covers the world, and its slender threads communicate themselves by degrees to the planets and stars. Captive now upon earth, I commune with the chorus of the stars who share in my joys and sorrows.”
—Gérard De Nerval (1808–1855)
“Can a free people restrain crime without sacrificing fundamental liberties and a heritage of compassion?... Let us show that we can temper together those opposite elements of liberty and restraint into one consistent whole. Let us set an example for the world of a law-abiding America glorying in its freedom as well as its respect for law.”
—Gerald R. Ford (b. 1913)
“The parents who wish to lead a quiet life I would say: Tell your children that they are very naughty—much naughtier than most children; point to the young people of some acquaintances as models of perfection, and impress your own children with a deep sense of their own inferiority. You carry so many more guns than they do that they cannot fight you. This is called moral influence and it will enable you to bounce them as much as you please.”
—Samuel Butler (1835–1902)