Bias of An Estimator - Bias, Variance and Mean Squared Error

Bias, Variance and Mean Squared Error

While bias quantifies the average difference to be expected between an estimator and an underlying parameter, an estimator based on a finite sample can additionally be expected to differ from the parameter due to the randomness in the sample.

One measure which is used to try to reflect both types of difference is the expected mean square error,

This can be shown to be equal to the square of the expected bias, plus the expected variance:

\begin{align} \operatorname{MSE}(\hat{\theta})= & (\operatorname{E})^2 + \operatorname{E})^2\,]\\ = & (\operatorname{Bias}(\hat{\theta},\theta))^2 + \operatorname{Var}(\hat{\theta}) \end{align}

An estimator that minimises the bias will not necessarily minimise the expected mean square error.

Read more about this topic:  Bias Of An Estimator

Famous quotes containing the words variance, squared and/or error:

    “There is an untroubled harmony in everything, a full consonance in nature; only in our illusory freedom do we feel at variance with it.”
    —Fyodor Tyutchev (1803–1873)

    “I want that head so sanitary and squared away that the Virgin Mary herself would be proud to go in there and take a dump.”
    —Stanley Kubrick (b. 1928)

    “Custom calls me to’t.
    What custom wills, in all things should we do’t,
    The dust on antique time would lie unswept,
    And mountainous error be too highly heaped
    For truth to o’erpeer.”
    —William Shakespeare (1564–1616)