AEP For Discrete-time I.i.d. Sources
Given is an i.i.d. source, its time series X1, ..., Xn is i.i.d. with entropy H(X) in the discrete-valued case and differential entropy in the continuous-valued case. The weak law of large numbers gives the AEP with convergence in probability,
since the entropy is equal to the expectation of . The strong law of large number asserts the stronger almost sure convergence,
which implies the result from the weak law of large numbers.
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the word sources:
“My profession brought me in contact with various minds. Earnest, serious discussion on the condition of woman enlivened my business room; failures of banks, no dividends from railroads, defalcations of all kinds, public and private, widows and orphans and unmarried women beggared by the dishonesty, or the mismanagement of men, were fruitful sources of conversation; confidence in man as a protector was evidently losing ground, and women were beginning to see that they must protect themselves.”
—Harriot K. Hunt (1805–1875)