Asymptotic Equipartition Property - AEP For Discrete-time I.i.d. Sources

AEP For Discrete-time I.i.d. Sources

Given is an i.i.d. source, its time series X₁, ..., X_n is i.i.d. with entropy H(X) in the discrete-valued case and differential entropy in the continuous-valued case. The weak law of large numbers gives the AEP with convergence in probability,

$\lim_{n\to\infty}\Pr\left=0 \qquad \forall \epsilon>0.$

since the entropy is equal to the expectation of . The strong law of large number asserts the stronger almost sure convergence,

$\Pr\left=1$

which implies the result from the weak law of large numbers.

Read more about this topic: Asymptotic Equipartition Property

Famous quotes containing the word sources:

“I count him a great man who inhabits a higher sphere of thought, into which other men rise with labor and difficulty; he has but to open his eyes to see things in a true light, and in large relations; whilst they must make painful corrections, and keep a vigilant eye on many sources of error.”
—Ralph Waldo Emerson (1803–1882)