Asymptotic Equipartition Property - AEP For Discrete-time I.i.d. Sources

AEP For Discrete-time I.i.d. Sources

Given is an i.i.d. source, its time series X1, ..., Xn is i.i.d. with entropy H(X) in the discrete-valued case and differential entropy in the continuous-valued case. The weak law of large numbers gives the AEP with convergence in probability,

\lim_{n\to\infty}\Pr\left=0 \qquad \forall \epsilon>0.

since the entropy is equal to the expectation of . The strong law of large number asserts the stronger almost sure convergence,

\Pr\left=1

which implies the result from the weak law of large numbers.

Read more about this topic:  Asymptotic Equipartition Property

Famous quotes containing the word sources:

    “The sources of poetry are in the spirit seeking completeness.”
    —Muriel Rukeyser (1913–1980)