Bayes Factor - Example

Example

Suppose we have a random variable which produces either a success or a failure. We want to compare a model M₁ where the probability of success is q = ½, and another model M₂ where q is completely unknown and we take a prior distribution for q which is uniform on . We take a sample of 200, and find 115 successes and 85 failures. The likelihood can be calculated according to the binomial distribution:

So we have

but

The ratio is then 1.197..., which is "barely worth mentioning" even if it points very slightly towards M₁.

This is not the same as a classical likelihood ratio test, which would have found the maximum likelihood estimate for q, namely 115⁄₂₀₀ = 0.575, and used that to get a ratio of 0.1045... (rather than averaging over all possible q), and so pointing towards M₂. Alternatively, Edwards's "exchange rate" of two units of likelihood per degree of freedom suggests that is preferable (just) to, as and : the extra likelihood compensates for the unknown parameter in .

A frequentist hypothesis test of (here considered as a null hypothesis) would have produced a more dramatic result, saying that M₁ could be rejected at the 5% significance level, since the probability of getting 115 or more successes from a sample of 200 if q = ½ is 0.0200..., and as a two-tailed test of getting a figure as extreme as or more extreme than 115 is 0.0400... Note that 115 is more than two standard deviations away from 100.

M₂ is a more complex model than M₁ because it has a free parameter which allows it to model the data more closely. The ability of Bayes factors to take this into account is a reason why Bayesian inference has been put forward as a theoretical justification for and generalisation of Occam's razor, reducing Type I errors.