Relation To Mutual Information
For analysis of contingency tables the value of G can also be expressed in terms of mutual information.
Let
- , and
Then G can be expressed in several alternative forms:
where the entropy of a discrete random variable is defined as
and where
is the mutual information between the row vector and the column vector of the contingency table.
It can also be shown that the inverse document frequency weighting commonly used for text retrieval is an approximation of G applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the G statistic.
Read more about this topic: G-test
Famous quotes containing the words relation to, relation, mutual and/or information:
“The proper study of mankind is man in his relation to his deity.”
—D.H. (David Herbert)
“It would be disingenuous, however, not to point out that some things are considered as morally certain, that is, as having sufficient certainty for application to ordinary life, even though they may be uncertain in relation to the absolute power of God.”
—René Descartes (1596–1650)
“I anticipate with pleasing expectations that retreat in which I promise myself to realize, without alloy, the sweet enjoyment of partaking, in the midst of my fellow citizens, the benign influence of good laws under a free government, the ever favorite object of my heart, and the happy reward, as I trust, of our mutual cares, labors, and dangers.”
—George Washington (1732–1799)
“I was brought up to believe that the only thing worth doing was to add to the sum of accurate information in the world.”
—Margaret Mead (1901–1978)