AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“There is no such source of error as the pursuit of absolute truth.”
—Samuel Butler (1835–1902)
“Syntax is the study of the principles and processes by which sentences are constructed in particular languages. Syntactic investigation of a given language has as its goal the construction of a grammar that can be viewed as a device of some sort for producing the sentences of the language under analysis.”
—Noam Chomsky (b. 1928)
“When the object is perceived as particular and unique and not merely the member of a family, when it appears independent of any general notion and detached from the sanity of a cause, isolated and inexplicable in the light of ignorance, then and only then may it be a source of enchantment.”
—Samuel Beckett (1906–1989)
“And into the gulf between cantankerous reality and the male ideal of shaping your world, sail the innocent children. They are right there in front of us—wild, irresponsible symbols of everything else we can’t control.”
—Hugh O’Neill (20th century)