Akaike Information Criterion - Bayesian Information Criterion

Bayesian Information Criterion

The AIC penalizes the number of parameters less strongly than does the Bayesian information criterion (BIC), which was independently developed by Akaike and by Schwarz in 1978, using Bayesian formalism. Akaike's version of BIC was originally denoted ABIC (for "a Bayesian Information Criterion") or referred to as Akaike's Bayesian Information Criterion.

A comparison of AIC/AICc and BIC is given by Burnham & Anderson (2002, § 6.4). The authors argue that AIC/AICc has theoretical advantages over BIC. Firstly, because AIC/AICc is derived from principles of information. Secondly, because the (Bayesian) derivation of BIC has a prior of 1/R (where R is the number of candidate models), which is "not sensible", since the prior should be a decreasing function of k. The authors also show that AIC and AICc can be derived in the same Bayesian framework as BIC, just by using a different prior. Additionally, they present a few simulation studies that suggest AICc tends to have practical/performance advantages over BIC. See also Burnham & Anderson (2004).

Further comparison of AIC and BIC, in the context of regression, is given by Yang (2005). In particular, AIC is asymptotically optimal in selecting the model with the least mean squared error, under the assumption that the exact "true" model is not in the candidate set (as is virtually always the case in practice); BIC is not asymptotically optimal under the assumption. Yang further shows that the rate at which AIC converges to the optimum is, in a certain sense, the best possible.