In information theory, the typical set is a set of sequences whose probability is close to two raised to the negative power of the entropy of their source distribution. That this set has total probability close to one is a consequence of the asymptotic equipartition property (AEP) which is a kind of law of large numbers. The notion of typicality is only concerned with the probability of a sequence and not the actual sequence itself.
This has great use in compression theory as it provides a theoretical means for compressing data, allowing us to represent any sequence Xn using nH(X) bits on average, and, hence, justifying the use of entropy as a measure of information from a source.
The AEP can also be proven for a large class of stationary ergodic processes, allowing typical set to be defined in more general cases.
Read more about Typical Set: (Weakly) Typical Sequences (weak Typicality, Entropy Typicality), Strongly Typical Sequences (strong Typicality, Letter Typicality), Jointly Typical Sequences
Famous quotes containing the words typical and/or set:
“Consciousness is cerebral celebrity—nothing more and nothing less. Those contents are conscious that persevere, that monopolize resources long enough to achieve certain typical and “symptomatic” effects—on memory, on the control of behavior and so forth.”
—Daniel Clement Dennett (b. 1942)
“Take two kids in competition for their parents’ love and attention. Add to that the envy that one child feels for the accomplishments of the other; the resentment that each child feels for the privileges of the other; the personal frustrations that they don’t dare let out on anyone else but a brother or sister, and it’s not hard to understand why in families across the land, the sibling relationship contains enough emotional dynamite to set off rounds of daily explosions.”
—Adele Faber (20th century)