Entropy In Thermodynamics And Information Theory
There are close parallels between the mathematical expressions for the thermodynamic entropy, usually denoted by S, of a physical system in the statistical thermodynamics established by Ludwig Boltzmann and J. Willard Gibbs in the 1870s, and the information-theoretic entropy, usually expressed as H, of Claude Shannon and Ralph Hartley developed in the 1940s. Shannon, although not initially aware of this similarity, commented on it upon publicizing information theory in A Mathematical Theory of Communication.
This article explores what links there are between the two concepts, and how far they can be regarded as connected.
Read more about Entropy In Thermodynamics And Information Theory: Theoretical Relationship, Negentropy, Black Holes, Quantum Theory, The Fluctuation Theorem
Famous quotes containing the words entropy, information and/or theory:
“Just as the constant increase of entropy is the basic law of the universe, so it is the basic law of life to be ever more highly structured and to struggle against entropy.”
—Václav Havel (b. 1936)
“I have all my life been on my guard against the information conveyed by the sense of hearing—it being one of my earliest observations, the universal inclination of humankind is to be led by the ears, and I am sometimes apt to imagine that they are given to men as they are to pitchers, purposely that they may be carried about by them.”
—Mary Wortley, Lady Montagu (1689–1762)
“The theory seems to be that so long as a man is a failure he is one of God’s chillun, but that as soon as he has any luck he owes it to the Devil.”
—H.L. (Henry Lewis)