AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“There often seems to be a playfulness to wise people, as if either their equanimity has as its source this playfulness or the playfulness flows from the equanimity; and they can persuade other people who are in a state of agitation to calm down and manage a smile.”
—Edward Hoagland (b. 1932)
“We have no more right to consume happiness without producing it than to consume wealth without producing it.”
—George Bernard Shaw (1856–1950)
“The chimney is to some extent an independent structure, standing on the ground, and rising through the house to the heavens; even after the house is burned it still stands sometimes, and its importance and independence are apparent.”
—Henry David Thoreau (1817–1862)
“Children became an obsessive theme in Victorian culture at the same time that they were being exploited as never before. As the horrors of life multiplied for some children, the image of childhood was increasingly exalted. Children became the last symbols of purity in a world which was seen as increasingly ugly.”
—C. John Sommerville (20th century)