Related Concepts
MDL is very strongly connected to probability theory and statistics through the correspondence between codes and probability distributions mentioned above. This has led researchers such as David MacKay to view MDL as equivalent to Bayesian inference: code length of the model and code length of model and data together in MDL correspond to prior probability and marginal likelihood respectively in the Bayesian framework.
While Bayesian machinery is often useful in constructing efficient MDL codes, the MDL framework also accommodates other codes that are not Bayesian. An example is the Shtarkov normalized maximum likelihood code, which plays a central role in current MDL theory, but has no equivalent in Bayesian inference. Furthermore, Rissanen stresses that we should make no assumptions about the true data generating process: in practice, a model class is typically a simplification of reality and thus does not contain any code or probability distribution that is true in any objective sense. In the last mentioned reference Rissanen bases the mathematical underpinning of MDL on the Kolmogorov structure function.
According to the MDL philosophy, Bayesian methods should be dismissed if they are based on unsafe priors that would lead to poor results. The priors that are acceptable from an MDL point of view also tend to be favored in so-called objective Bayesian analysis; there, however, the motivation is usually different.
Read more about this topic: Minimum Description Length
Famous quotes containing the words related and/or concepts:
“Generally there is no consistent evidence of significant differences in school achievement between children of working and nonworking mothers, but differences that do appear are often related to maternal satisfaction with her chosen role, and the quality of substitute care.”
—Ruth E. Zambrana, U.S. researcher, M. Hurst, and R.L. Hite. “The Working Mother in Contemporary Perspectives: A Review of Literature,” Pediatrics (December 1979)
“Institutional psychiatry is a continuation of the Inquisition. All that has really changed is the vocabulary and the social style. The vocabulary conforms to the intellectual expectations of our age: it is a pseudo-medical jargon that parodies the concepts of science. The social style conforms to the political expectations of our age: it is a pseudo-liberal social movement that parodies the ideals of freedom and rationality.”
—Thomas Szasz (b. 1920)