Relation To Mutual Information
For analysis of contingency tables the value of G can also be expressed in terms of mutual information.
Let
- , and
Then G can be expressed in several alternative forms:
where the entropy of a discrete random variable is defined as
and where
is the mutual information between the row vector and the column vector of the contingency table.
It can also be shown that the inverse document frequency weighting commonly used for text retrieval is an approximation of G applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the G statistic.
Read more about this topic: G-test
Famous quotes containing the words relation to, relation, mutual and/or information:
“Whoever has a keen eye for profits, is blind in relation to his craft.”
—Sophocles (497–406/5 B.C.)
“There is a constant in the average American imagination and taste, for which the past must be preserved and celebrated in full-scale authentic copy; a philosophy of immortality as duplication. It dominates the relation with the self, with the past, not infrequently with the present, always with History and, even, with the European tradition.”
—Umberto Eco (b. 1932)
“For do but note a wild and wanton herd
Or race of youthful and unhandled colts
Fetching mad bounds, bellowing and neighing loud,
Which is the hot condition of their blood;
If they but hear perchance a trumpet sound,
Or any air of music touch their ears,
You shall perceive them make a mutual stand,
Their savage eyes turned to a modest gaze
By the sweet power of music.”
—William Shake{peare (1564–1616)
“So while it is true that children are exposed to more information and a greater variety of experiences than were children of the past, it does not follow that they automatically become more sophisticated. We always know much more than we understand, and with the torrent of information to which young people are exposed, the gap between knowing and understanding, between experience and learning, has become even greater than it was in the past.”
—David Elkind (20th century)