Bias of An Estimator - Bias, Variance and Mean Squared Error

Bias, Variance and Mean Squared Error

While bias quantifies the average difference to be expected between an estimator and an underlying parameter, an estimator based on a finite sample can additionally be expected to differ from the parameter due to the randomness in the sample.

One measure which is used to try to reflect both types of difference is the expected mean square error,

This can be shown to be equal to the square of the expected bias, plus the expected variance:

\begin{align} \operatorname{MSE}(\hat{\theta})= & (\operatorname{E})^2 + \operatorname{E})^2\,]\\ = & (\operatorname{Bias}(\hat{\theta},\theta))^2 + \operatorname{Var}(\hat{\theta}) \end{align}

An estimator that minimises the bias will not necessarily minimise the expected mean square error.

Read more about this topic:  Bias Of An Estimator

Famous quotes containing the words variance, squared and/or error:

    “There is an untroubled harmony in everything, a full consonance in nature; only in our illusory freedom do we feel at variance with it.”
    —Fyodor Tyutchev (1803–1873)

    “Dreams are toys.
    Yet for this once, yea, superstitiously,
    I will be squared by this.”
    —William Shakespeare (1564–1616)

    “It is the very error of the moon,
    She comes more near the earth than she was wont,
    And makes men mad.”
    —William Shakespeare (1564–1616)