AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“The source of poetry that
seeing the clock stopped, says,
The clock has stopped
that ticked yesterday so well?”
—William Carlos Williams (1883–1963)
“Death is an incident producing clay. Use it, mold it, learn from it.”
—John Gilling, British screenwriter. Dr. Knox (Peter Cushing)
“The dignity and stability of government in all its branches, the morals of the people, and every blessing of society, depend so much upon an upright and skilful administration of justice, that the judicial power ought to be distinct from both the legislative and executive, and independent upon both, that so it may be a check upon both, as both should be checks upon that.”
—Thomas Jefferson (1743–1826)
“My image is a statement of the symbols of the harsh, impersonal products and brash materialistic objects on which America is built today. It is a projection of everything that can be bought and sold, the practical but impermanent symbols that sustain us.”
—Andy Warhol (1928–1987)