Definition For Simple Linear Regression of Data Points
This is the definition of regression toward the mean that closely follows Sir Francis Galton's original usage.
Suppose there are n data points {yi, xi}, where i = 1, 2, …, n. We want to find the equation of the regression line, i.e. the straight line
which would provide a “best” fit for the data points. (Note that a straight line may not be the appropriate regression curve for the given data points.) Here the “best” will be understood as in the least-squares approach: such a line that minimizes the sum of squared residuals of the linear regression model. In other words, numbers α and β solve the following minimization problem:
- Find, where
Using simple calculus it can be shown that the values of α and β that minimize the objective function Q are
where rxy is the sample correlation coefficient between x and y, sx is the standard deviation of x, and sy is correspondingly the standard deviation of y. Horizontal bar over a variable means the sample average of that variable. For example:
Substituting the above expressions for and into yields fitted values
which yields
This shows the role rxy plays in the regression line of standardized data points.
If −1 < rxy < 1, then we say that the data points exhibit regression toward the mean. In other words, if linear regression is the appropriate model for a set of data points whose sample correlation coefficient is not perfect, then there is regression toward the mean. The predicted (or fitted) standardized value of y is closer to its mean than the standardized value of x is to its mean.
Read more about this topic: Regression Toward The Mean
Famous quotes containing the words definition, simple, data and/or points:
“The definition of good prose is proper words in their proper places; of good verse, the most proper words in their proper places. The propriety is in either case relative. The words in prose ought to express the intended meaning, and no more; if they attract attention to themselves, it is, in general, a fault.”
—Samuel Taylor Coleridge (17721834)
“A simple and independent mind does not toil at the bidding of any prince.”
—Henry David Thoreau (18171862)
“To write it, it took three months; to conceive it three minutes; to collect the data in itall my life.”
—F. Scott Fitzgerald (18961940)
“We only part to meet again.
Change, as ye list, ye winds: my heart shall be
The faithful compass that still points to thee.”
—John Gay (16851732)