AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“We are constantly railing against the passions; we ascribe to them all of man’s afflictions, and we forget that they are also the source of all his pleasures.”
—Denis Diderot (1713–1784)
“Sir, I do not call a gamester a dishonest man; but I call him an unsocial man, an unprofitable man. Gaming is a mode of transferring property without producing any intermediate good.”
—Samuel Johnson (1709–1784)
“There is in fact no such thing as art for art’s sake, art that stands above classes, art that is detached from or independent of politics. Proletarian literature and art are part of the whole proletarian revolutionary cause.”
—Mao Zedong (1893–1976)
“Many older wealthy families have learned to instill a sense of public service in their offspring. But newly affluent middle-class parents have not acquired this skill. We are using our children as symbols of leisure-class standing without building in safeguards against an overweening sense of entitlement—a sense of entitlement that may incline some young people more toward the good life than toward the hard work that, for most of us, makes the good life possible.”
—David Elkind (20th century)