Minimum Mean Square Error

In statistics and signal processing, a minimum mean square error (MMSE) estimator describes the approach which minimizes the mean square error (MSE), which is a common measure of estimator quality.

The term MMSE more specifically refers to estimation in a Bayesian setting with quadratic cost function. The idea is that we often have some prior information about the parameters to be estimated, instead of knowing absolutely nothing about it. This prior information is captured by the prior probability density function of the parameters and allows us to make better posterior estimates as more observations become available. Thus unlike non-Bayesian approach where parameters of interest are assumed to be deterministic, but unknown constants, the Bayesian estimator seeks to estimate a parameter that is itself a random variable. The Bayesian approach, based directly on Bayes’ theorem, provides a framework for handling such problems by allowing prior knowledge to be incorporated into the estimator. Furthermore, Bayesian estimation provides yet another alternative to the minimum-variance unbiased estimator (MVUE). This is useful when the MVUE cannot be found.

In the alternative frequentist setting there does not exist a single estimator having minimal MSE. A somewhat similar concept can be obtained within the frequentist point of view if one requires unbiasedness, since an estimator may exist that minimizes the variance (and hence the MSE) among unbiased estimators. Such an estimator is then called the MVUE.