History
The Boltzmann machine is a Monte Carlo version of the Hopfield network.
The idea of using annealed Ising models for inference is often thought to have been first described by:
- Geoffrey E. Hinton and Terrence J. Sejnowski, Analyzing Cooperative Computation. In Proceedings of the 5th Annual Congress of the Cognitive Science Society, Rochester, NY, May 1983.
- Geoffrey E. Hinton and Terrence J. Sejnowski, Optimal Perceptual Inference. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), pages 448–453, IEEE Computer Society, Washington DC, June 1983.
However, it should be noted that these articles appeared after the seminal publication by John Hopfield, where the connection to physics and statistical mechanics was made in the first place, mentioning spin glasses:
- John J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Natl. Acad. Sci. USA, vol. 79 no. 8, pp. 2554-2558, April 1982.
The idea of applying the Ising model with annealed Gibbs sampling is also present in Douglas Hofstadter's Copycat project:
- Hofstadter, Douglas R., The Copycat Project: An Experiment in Nondeterminism and Creative Analogies. MIT Artificial Intelligence Laboratory Memo No. 755, January 1984.
- Hofstadter, Douglas R., A Non-Deterministic Approach to Analogy, Involving the Ising Model of Ferromagnetism. In E. Caianiello, ed. The Physics of Cognitive Processes. Teaneck, NJ: World Scientific, 1987.
Similar ideas (with a change of sign in the energy function) are also found in Paul Smolensky's "Harmony Theory".
The explicit analogy drawn with statistical mechanics in the Boltzmann Machine formulation led to the use of terminology borrowed from physics (e.g., "energy" rather than "harmony"), which has become standard in the field. The widespread adoption of this terminology may have been encouraged by the fact that its use led to the importation of a variety of concepts and methods from statistical mechanics. However, there is no reason to think that the various proposals to use simulated annealing for inference described above were not independent. (Helmholtz made a similar analogy during the dawn of psychophysics.)
Ising models are now considered to be a special case of Markov random fields, which find widespread application in various fields, including linguistics, robotics, computer vision, and artificial intelligence.
Read more about this topic: Boltzmann Machine
Famous quotes containing the word history:
“History does nothing; it does not possess immense riches, it does not fight battles. It is men, real, living, who do all this.... It is not “history” which uses men as a means of achieving—as if it were an individual person—its own ends. History is nothing but the activity of men in pursuit of their ends.”
—Karl Marx (1818–1883)
“Whenever we read the obscene stories, the voluptuous debaucheries, the cruel and torturous executions, the unrelenting vindictiveness, with which more than half the Bible is filled, it would be more consistent that we called it the word of a demon than the Word of God. It is a history of wickedness that has served to corrupt and brutalize mankind.”
—Thomas Paine (1737–1809)
“We may pretend that we’re basically moral people who make mistakes, but the whole of history proves otherwise.”
—Terry Hands (b. 1941)