AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“The belief in a supernatural source of evil is not necessary; men alone are quite capable of every wickedness.”
—Joseph Conrad (1857–1924)
“What ails it, intrinsically, is a dearth of intellectual audacity and of aesthetic passion. Running through it, and characterizing the work of almost every man and woman producing it, there is an unescapable suggestion of the old Puritan suspicion of the fine arts as such—of the doctrine that they offer fit asylum for good citizens only when some ulterior and superior purpose is carried into them.”
—H.L. (Henry Lewis)
“[My father] was a lazy man. It was the days of independent incomes, and if you had an independent income you didn’t work. You weren’t expected to. I strongly suspect that my father would not have been particularly good at working anyway. He left our house in Torquay every morning and went to his club. He returned, in a cab, for lunch, and in the afternoon went back to the club, played whist all afternoon, and returned to the house in time to dress for dinner.”
—Agatha Christie (1891–1976)
“If the Americans, in addition to the eagle and the Stars and Stripes and the more unofficial symbols of bison, moose and Indian, should ever need another emblem, one which is friendly and pleasant, then I think they should choose the grapefruit. Or rather the half grapefruit, for this fruit only comes in halves, I believe. Practically speaking, it is always yellow, always just as fresh and well served. And it always comes at the same, still hopeful hour of the morning.”
—Johan Huizinga (1872–1945)