Bayesian Brain - Origins

Origins

This field of study has its historical roots in numerous disciplines including machine learning, experimental psychology and Bayesian statistics. As early as the 1860s, with the work of Hermann Helmholtz in experimental psychology the brain's ability to extract perceptual information from sensory data was modeled in terms of probabilistic estimation. The basic idea is that the nervous system needs to organize sensory data into an accurate internal model of the outside world.

Bayesian probability has been developed by many important contributors. Pierre-Simon Laplace, Thomas Bayes, Harold Jeffreys, Richard Cox and Edwin Jaynes developed mathematical techniques and procedures for treating probability as the degree of plausibility that could be assigned to a given supposition or hypothesis based on the available evidence. In 1988 E.T. Jaynes presented a framework for using Bayesian Probability to model mental processes. It was thus realized early on that the Bayesian statistical framework holds the potential to lead to insights into the function of the nervous system.

This idea was taken up in research on unsupervised learning, in particular the Analysis by Synthesis approach, branches of machine learning. In 1983 Geoffrey Hinton and colleagues proposed the brain could be seen as a machine making decisions based on the uncertainties of the outside world. During the 1990s researchers including Peter Dayan, Geoffrey Hinton and Richard Zemel proposed that the brain represents knowledge of the world in terms of probabilities and made specific proposals for tractable neural processes that could manifest such a Helmholtz Machine.