In information theory, the typical set is a set of sequences whose probability is close to two raised to the negative power of the entropy of their source distribution. That this set has total probability close to one is a consequence of the asymptotic equipartition property (AEP) which is a kind of law of large numbers. The notion of typicality is only concerned with the probability of a sequence and not the actual sequence itself.
This has great use in compression theory as it provides a theoretical means for compressing data, allowing us to represent any sequence Xn using nH(X) bits on average, and, hence, justifying the use of entropy as a measure of information from a source.
The AEP can also be proven for a large class of stationary ergodic processes, allowing typical set to be defined in more general cases.
Read more about Typical Set: (Weakly) Typical Sequences (weak Typicality, Entropy Typicality), Strongly Typical Sequences (strong Typicality, Letter Typicality), Jointly Typical Sequences
Famous quotes containing the words typical and/or set:
“It is not however, adulthood itself, but parenthood that forms the glass shroud of memory. For there is an interesting quirk in the memory of women. At 30, women see their adolescence quite clearly. At 30 a woman’s adolescence remains a facet fitting into her current self.... At 40, however, memories of adolescence are blurred. Women of this age look much more to their earlier childhood for memories of themselves and of their mothers. This links up to her typical parenting phase.”
—Terri Apter (20th century)
“Time doth transfix the flourish set on youth
And delves the parallels in beauty’s brow,
Feeds on the rarities of nature’s truth,
And nothing stands but for his scythe to mow:
And yet to times in hope my verse shall stand,
Praising thy worth, despite his cruel hand.”
—William Shakespeare (1564–1616)