In information theory, the typical set is a set of sequences whose probability is close to two raised to the negative power of the entropy of their source distribution. That this set has total probability close to one is a consequence of the asymptotic equipartition property (AEP) which is a kind of law of large numbers. The notion of typicality is only concerned with the probability of a sequence and not the actual sequence itself.
This has great use in compression theory as it provides a theoretical means for compressing data, allowing us to represent any sequence Xn using nH(X) bits on average, and, hence, justifying the use of entropy as a measure of information from a source.
The AEP can also be proven for a large class of stationary ergodic processes, allowing typical set to be defined in more general cases.
Read more about Typical Set: (Weakly) Typical Sequences (weak Typicality, Entropy Typicality), Strongly Typical Sequences (strong Typicality, Letter Typicality), Jointly Typical Sequences
Famous quotes containing the words typical and/or set:
“Sinclair Lewis is the perfect example of the false sense of time of the newspaper world.... [ellipsis in source] He was always dominated by an artificial time when he wrote Main Street.... He did not create actual human beings at any time. That is what makes it newspaper. Sinclair Lewis is the typical newspaperman and everything he says is newspaper. The difference between a thinker and a newspaperman is that a thinker enters right into things, a newspaperman is superficial.”
—Gertrude Stein (1874–1946)
“Through dinner she felt a gradual icy coldness stealing through her like novocaine. She had made up her mind. It seemed as if she had set the photograph of herself in her own place, forever frozen into a single gesture.”
—John Dos Passos (1896–1970)