Historical Motivation
The method of least squares is a prototypical M-estimator, since the estimator is defined as a minimum of the sum of squares of the residuals.
Another popular M-estimator is maximum-likelihood estimation. For a family of probability density functions f parameterized by θ, a maximum likelihood estimator of θ is computed for each set of data by maximizing the likelihood function over the parameter space { θ } . When the observations are independent and identically distributed, a ML-estimate satisfies
or, equivalently,
Maximum-likelihood estimators are often inefficient and biased for finite samples. For many regular problems, maximum-likelihood estimation performs well for "large samples", being an approximation of a posterior mode. If the problem is "regular", then any bias of the MLE (or posterior mode) decreases to zero when the sample-size increases to infinity. The performance of maximum-likelihood (and posterior-mode) estimators drops when the parametric family is mis-specified.
Read more about this topic: M-estimator
Famous quotes containing the words historical and/or motivation:
“Some of us still get all weepy when we think about the Gaia Hypothesis, the idea that earth is a big furry goddess-creature who resembles everybody’s mom in that she knows what’s best for us. But if you look at the historical record—Krakatoa, Mt. Vesuvius, Hurricane Charley, poison ivy, and so forth down the ages—you have to ask yourself: Whose side is she on, anyway?”
—Barbara Ehrenreich (b. 1941)
“Self-determination has to mean that the leader is your individual gut, and heart, and mind or we’re talking about power, again, and its rather well-known impurities. Who is really going to care whether you live or die and who is going to know the most intimate motivation for your laughter and your tears is the only person to be trusted to speak for you and to decide what you will or will not do.”
—June Jordan (b. 1939)