Entropy Rates For Markov Chains
Since a stochastic process defined by a Markov chain that is irreducible and aperiodic has a stationary distribution, the entropy rate is independent of the initial distribution.
For example, for such a Markov chain Yk defined on a countable number of states, given the transition matrix Pij, H(Y) is given by:
where μi is the stationary distribution of the chain.
A simple consequence of this definition is that the entropy rate of an i.i.d. stochastic process has an entropy rate that is the same as the entropy of any individual member of the process.
Read more about this topic: Entropy Rate
Famous quotes containing the words entropy, rates and/or chains:
“Just as the constant increase of entropy is the basic law of the universe, so it is the basic law of life to be ever more highly structured and to struggle against entropy.”
—Václav Havel (b. 1936)
“In the U.S. for instance, the value of a homemaker’s productive work has been imputed mostly when she was maimed or killed and insurance companies and/or the courts had to calculate the amount to pay her family in damages. Even at that, the rates were mostly pink collar and the big number was attributed to the husband’s pain and suffering.”
—Gloria Steinem (20th century)
“Let the saints be joyful in glory: let them sing aloud upon their beds. Let the high praises of God be in their mouth, and a two-edged sword in their hand; to execute vengeance upon the heathen, and punishments upon the people; to bind their kings with chains and their nobles with fetters of iron; to execute upon them the judgment written.”
—Bible: Hebrew Psalms 149:5-9.