Asymptotic Equipartition Property - Definition

Definition

Given a discrete-time stationary ergodic stochastic process on the probability space, AEP is an assertion that

-\frac{1}{n} \log p(X_1^n) \to H(X)
\quad \mbox{ as } \quad n\to\infty

where denotes the process limited to duration, and or simply denotes the entropy rate of, which must exist for all discrete-time stationary processes including the ergodic ones. AEP is proved for finite-valued (i.e. ) stationary ergodic stochastic processes in the Shannon-McMillan-Breiman theorem using the ergodic theory and for any i.i.d. sources directly using the law of large numbers in both the discrete-valued case (where is simply the entropy of a symbol) and the continuous-valued case (where is the differential entropy instead). The definition of AEP can also be extended for certain classes of continuous-time stochastic processes for which a typical set exists for long enough observation time. The convergence is proven almost sure in all cases.

Read more about this topic:  Asymptotic Equipartition Property

Famous quotes containing the word definition:

    I’m beginning to think that the proper definition of “Man” is “an animal that writes letters.”
    Lewis Carroll [Charles Lutwidge Dodgson] (1832–1898)

    No man, not even a doctor, ever gives any other definition of what a nurse should be than this—”devoted and obedient.” This definition would do just as well for a porter. It might even do for a horse. It would not do for a policeman.
    Florence Nightingale (1820–1910)

    It is very hard to give a just definition of love. The most we can say of it is this: that in the soul, it is a desire to rule; in the spirit, it is a sympathy; and in the body, it is but a hidden and subtle desire to possess—after many mysteries—what one loves.
    François, Duc De La Rochefoucauld (1613–1680)