Relation To Mutual Information
For analysis of contingency tables the value of G can also be expressed in terms of mutual information.
Let
- , and
Then G can be expressed in several alternative forms:
where the entropy of a discrete random variable is defined as
and where
is the mutual information between the row vector and the column vector of the contingency table.
It can also be shown that the inverse document frequency weighting commonly used for text retrieval is an approximation of G applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the G statistic.
Read more about this topic: G-test
Famous quotes containing the words relation to, relation, mutual and/or information:
“Among the most valuable but least appreciated experiences parenthood can provide are the opportunities it offers for exploring, reliving, and resolving one’s own childhood problems in the context of one’s relation to one’s child.”
—Bruno Bettelheim (20th century)
“There is a constant in the average American imagination and taste, for which the past must be preserved and celebrated in full-scale authentic copy; a philosophy of immortality as duplication. It dominates the relation with the self, with the past, not infrequently with the present, always with History and, even, with the European tradition.”
—Umberto Eco (b. 1932)
“Of course I lie to people. But I lie altruistically—for our mutual good. The lie is the basic building block of good manners. That may seem mildly shocking to a moralist—but then what isn’t?”
—Quentin Crisp (b. 1908)
“I believe it has been said that one copy of The Times contains more useful information than the whole of the historical works of Thucydides.”
—Richard Cobden (1804–1865)