Ordinary Least Squares - Finite Sample Properties

Finite Sample Properties

First of all, under the strict exogeneity assumption the OLS estimators and s2 are unbiased, meaning that their expected values coincide with the true values of the parameters:

$\operatorname{E} = \beta, \quad \operatorname{E} = \sigma^2.$

If the strict exogeneity does not hold (as is the case with many time series models, where exogeneity is assumed only with respect to the past shocks but not the future ones), then these estimators will be biased in finite samples.

The variance-covariance matrix of is equal to

$\operatorname{Var} = \sigma^2(X'X)^{-1}.$

In particular, the standard error of each coefficient is equal to square root of the j-th diagonal element of this matrix. The estimate of this standard error is obtained by replacing the unknown quantity σ2 with its estimate s2. Thus,

$\widehat{\operatorname{s.\!e}}(\hat{\beta}_j) = \sqrt{s^2 (X'X)^{-1}_{jj}}$

It can also be easily shown that the estimator is uncorrelated with the residuals from the model:

$\operatorname{Cov} = 0.$

The Gauss–Markov theorem states that under the spherical errors assumption (that is, the errors should be uncorrelated and homoscedastic) the estimator is efficient in the class of linear unbiased estimators. This is called the best linear unbiased estimator (BLUE). Efficiency should be understood as if we were to find some other estimator which would be linear in y and unbiased, then

$\operatorname{Var} - \operatorname{Var} \geq 0$

in the sense that this is a nonnegative-definite matrix. This theorem establishes optimality only in the class of linear unbiased estimators, which is quite restrictive. Depending on the distribution of the error terms ε, other, non-linear estimators may provide better results than OLS.

Read more about this topic: Ordinary Least Squares

Famous quotes containing the words finite, sample and/or properties:

“Put shortly, these are the two views, then. One, that man is intrinsically good, spoilt by circumstance; and the other that he is intrinsically limited, but disciplined by order and tradition to something fairly decent. To the one party man’s nature is like a well, to the other like a bucket. The view which regards him like a well, a reservoir full of possibilities, I call the romantic; the one which regards him as a very finite and fixed creature, I call the classical.”
—Thomas Ernest Hulme (1883–1917)

“As a rule they will refuse even to sample a foreign dish, they regard such things as garlic and olive oil with disgust, life is unliveable to them unless they have tea and puddings.”
—George Orwell (1903–1950)

“A drop of water has the properties of the sea, but cannot exhibit a storm. There is beauty of a concert, as well as of a flute; strength of a host, as well as of a hero.”
—Ralph Waldo Emerson (1803–1882)