Asymptotic Equipartition Property - AEP For Discrete-time I.i.d. Sources

AEP For Discrete-time I.i.d. Sources

Given is an i.i.d. source, its time series X1, ..., Xn is i.i.d. with entropy H(X) in the discrete-valued case and differential entropy in the continuous-valued case. The weak law of large numbers gives the AEP with convergence in probability,

\lim_{n\to\infty}\Pr\left=0 \qquad \forall \epsilon>0.

since the entropy is equal to the expectation of . The strong law of large number asserts the stronger almost sure convergence,

\Pr\left=1

which implies the result from the weak law of large numbers.

Read more about this topic:  Asymptotic Equipartition Property

Famous quotes containing the word sources:

    “The American grips himself, at the very sources of his consciousness, in a grip of care: and then, to so much of the rest of life, is indifferent. Whereas, the European hasn’t got so much care in him, so he cares much more for life and living.”
    —D.H. (David Herbert)