Minimum Message Length
The minimum message length principle of statistical and inductive inference and machine learning was developed by C.S. Wallace and D.M. Boulton in 1968. MML is Bayesian (i.e. it incorporates prior beliefs) and information-theoretic. It has the desirable properties of statistical invariance (i.e. the inference transforms with a re-parametrisation, such as from polar coordinates to Cartesian coordinates), statistical consistency (i.e. even for very hard problems, MML will converge to any underlying model) and efficiency (i.e. the MML model will converge to any true underlying model about as quickly as is possible). C.S. Wallace and D.L. Dowe (1999) showed a formal connection between MML and algorithmic information theory (or Kolmogorov complexity).
Read more about this topic: Kolmogorov Complexity
Famous quotes containing the words minimum, message and/or length:
“There are ... two minimum conditions necessary and sufficient for the existence of a legal system. On the one hand those rules of behavior which are valid according to the system’s ultimate criteria of validity must be generally obeyed, and on the other hand, its rules of recognition specifying the criteria of legal validity and its rules of change and adjudication must be effectively accepted as common public standards of official behavior by its officials.”
—H.L.A. (Herbert Lionel Adolphus)
“Children in home-school conflict situations often receive a double message from their parents: “The school is the hope for your future, listen, be good and learn” and “the school is your enemy. . . .” Children who receive the “school is the enemy” message often go after the enemy—act up, undermine the teacher, undermine the school program, or otherwise exercise their veto power.”
—James P. Comer (20th century)
“You will, I am sure, agree with me that ... if page 534 only finds us in the second chapter, the length of the first one must have been really intolerable.”
—Sir Arthur Conan Doyle (1859–1930)