Maximum A Posteriori Estimation - Description

Description

Assume that we want to estimate an unobserved population parameter on the basis of observations . Let be the sampling distribution of, so that is the probability of when the underlying population parameter is . Then the function

is known as the likelihood function and the estimate

is the maximum likelihood estimate of .

Now assume that a prior distribution over exists. This allows us to treat as a random variable as in Bayesian statistics. Then the posterior distribution of is as follows:

where is density function of, is the domain of . This is a straightforward application of Bayes' theorem.

The method of maximum a posterior estimation then estimates as the mode of the posterior distribution of this random variable:

\hat{\theta}_{\mathrm{MAP}}(x) = \underset{\theta}{\operatorname{arg\,max}} \ \frac{f(x | \theta) \, g(\theta)} {\displaystyle\int_{\vartheta} f(x | \vartheta) \, g(\vartheta) \, d\vartheta} = \underset{\theta}{\operatorname{arg\,max}} \ f(x | \theta) \, g(\theta). \!

The denominator of the posterior distribution (so-called partition function) does not depend on and therefore plays no role in the optimization. Observe that the MAP estimate of coincides with the ML estimate when the prior is uniform (that is, a constant function). The MAP estimate is a limit of Bayes estimators under a sequence of 0-1 loss functions, but generally not a Bayes estimator per se, unless is discrete.

Read more about this topic:  Maximum A Posteriori Estimation

Famous quotes containing the word description:

    “As they are not seen on their way down the streams, it is thought by fishermen that they never return, but waste away and die, clinging to rocks and stumps of trees for an indefinite period; a tragic feature in the scenery of the river bottoms worthy to be remembered with Shakespeare’s description of the sea-floor.”
    —Henry David Thoreau (1817–1862)

    “Once a child has demonstrated his capacity for independent functioning in any area, his lapses into dependent behavior, even though temporary, make the mother feel that she is being taken advantage of....What only yesterday was a description of the child’s stage in life has become an indictment, a judgment.”
    —Elaine Heffner (20th century)

    “The great object in life is Sensation—to feel that we exist, even though in pain; it is this “craving void” which drives us to gaming, to battle, to travel, to intemperate but keenly felt pursuits of every description whose principal attraction is the agitation inseparable from their accomplishment.”
    —George Gordon Noel Byron (1788–1824)