Entropy Rates For Markov Chains
Since a stochastic process defined by a Markov chain that is irreducible and aperiodic has a stationary distribution, the entropy rate is independent of the initial distribution.
For example, for such a Markov chain Yk defined on a countable number of states, given the transition matrix Pij, H(Y) is given by:
where μi is the stationary distribution of the chain.
A simple consequence of this definition is that the entropy rate of an i.i.d. stochastic process has an entropy rate that is the same as the entropy of any individual member of the process.
Read more about this topic: Entropy Rate
Famous quotes containing the words entropy, rates and/or chains:
“Just as the constant increase of entropy is the basic law of the universe, so it is the basic law of life to be ever more highly structured and to struggle against entropy.”
—Václav Havel (b. 1936)
“[The] elderly and timid single gentleman in Paris ... never drove down the Champs Elysees without expecting an accident, and commonly witnessing one; or found himself in the neighborhood of an official without calculating the chances of a bomb. So long as the rates of progress held good, these bombs would double in force and number every ten years.”
—Henry Brooks Adams (1838–1918)
“... there are no chains so galling as the chains of ignorance—no fetters so binding as those that bind the soul, and exclude it from the vast field of useful and scientific knowledge. O, had I received the advantages of early education, my ideas would, ere now, have expanded far and wide; but, alas! I possess nothing but moral capability—no teachings but the teachings of the Holy Spirit.”
—Maria Stewart (1803–1879)