Simple Linear Regression - Fitting The Regression Line

Fitting The Regression Line

Suppose there are n data points {yi, xi}, where i = 1, 2, …, n. The goal is to find the equation of the straight line

which would provide a "best" fit for the data points. Here the "best" will be understood as in the least-squares approach: such a line that minimizes the sum of squared residuals of the linear regression model. In other words, numbers α and β solve the following minimization problem:

By using either calculus, the geometry of inner product spaces or simply expanding to get a quadratic in α and β, it can be shown that the values of α and β that minimize the objective function Q are

\begin{align} \hat\beta & = \frac{ \sum_{i=1}^{n} (x_{i}-\bar{x})(y_{i}-\bar{y}) }{ \sum_{i=1}^{n} (x_{i}-\bar{x})^2 } = \frac{ \sum_{i=1}^{n}{x_{i}y_{i}} - \frac1n \sum_{i=1}^{n}{x_{i}}\sum_{j=1}^{n}{y_{j}}}{ \sum_{i=1}^{n}({x_{i}^2}) - \frac1n (\sum_{i=1}^{n}{x_{i}})^2 } \\ & = \frac{ \overline{xy} - \bar{x}\bar{y} }{ \overline{x^2} - \bar{x}^2 } = \frac{ \operatorname{Cov} }{ \operatorname{Var} } = r_{xy} \frac{s_y}{s_x}, \\ \hat\alpha & = \bar{y} - \hat\beta\,\bar{x}, \end{align}

where rxy is the sample correlation coefficient between x and y, sx is the standard deviation of x, and sy is correspondingly the standard deviation of y. Horizontal bar over a variable means the sample average of that variable. For example:

Substituting the above expressions for and into

yields

This shows the role plays in the regression line of standardized data points.

Read more about this topic:  Simple Linear Regression

Famous quotes containing the words fitting and/or line:

    “The most fitting monuments this nation can build are schoolhouses and homes for those who do the work of the world. It is no answer to say that they are accustomed to rags and hunger. In this world of plenty every human being has a right to food, clothes, decent shelter, and the rudiments of education.”
    —Elizabeth Cady Stanton (1815–1902)

    “What comes over a man, is it soul or mind
    That to no limits and bounds he can stay confined?
    You would say his ambition was to extend the reach
    Clear to the Arctic of every living kind.
    Why is his nature forever so hard to teach
    That though there is no fixed line between wrong and right,
    There are roughly zones whose laws must be obeyed?”
    —Robert Frost (1874–1963)