Quantities of Information
Information theory is based on probability theory and statistics. The most important quantities of information are entropy, the information in a random variable, and mutual information, the amount of information in common between two random variables. The former quantity indicates how easily message data can be compressed while the latter can be used to find the communication rate across a channel.
The choice of logarithmic base in the following formulae determines the unit of information entropy that is used. The most common unit of information is the bit, based on the binary logarithm. Other units include the nat, which is based on the natural logarithm, and the hartley, which is based on the common logarithm.
In what follows, an expression of the form is considered by convention to be equal to zero whenever This is justified because for any logarithmic base.
Read more about this topic: Information Theory
Famous quotes containing the words quantities of, quantities and/or information:
“Compilers resemble gluttonous eaters who devour excessive quantities of healthy food just to excrete them as refuse.”
—Franz Grillparzer (1791–1872)
“James Brown and Frank Sinatra are two different quantities in the universe. They represent two different experiences of the world.”
—Imamu Amiri Baraka [Everett Leroi Jones] (b. 1934)
“Many more children observe attitudes, values and ways different from or in conflict with those of their families, social networks, and institutions. Yet today’s young people are no more mature or capable of handling the increased conflicting and often stimulating information they receive than were young people of the past, who received the information and had more adult control of and advice about the information they did receive.”
—James P. Comer (20th century)