Relation To Mutual Information
For analysis of contingency tables the value of G can also be expressed in terms of mutual information.
Let
- , and
Then G can be expressed in several alternative forms:
where the entropy of a discrete random variable is defined as
and where
is the mutual information between the row vector and the column vector of the contingency table.
It can also be shown that the inverse document frequency weighting commonly used for text retrieval is an approximation of G applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the G statistic.
Read more about this topic: G-test
Famous quotes containing the words relation to, relation, mutual and/or information:
“A theory of the middle class: that it is not to be determined by its financial situation but rather by its relation to government. That is, one could shade down from an actual ruling or governing class to a class hopelessly out of relation to government, thinking of gov’t as beyond its control, of itself as wholly controlled by gov’t. Somewhere in between and in gradations is the group that has the sense that gov’t exists for it, and shapes its consciousness accordingly.”
—Lionel Trilling (1905–1975)
“Art should exhilarate, and throw down the walls of circumstance on every side, awakening in the beholder the same sense of universal relation and power which the work evinced in the artist, and its highest effect is to make new artists.”
—Ralph Waldo Emerson (1803–1882)
“Whoever takes a view of the life of man ... will find it so beset and hemm’d in with obligations of one kind or other, as to leave little room to suspect, that man can live to himself: and so closely has our creator link’d us together ... that we find this bond of mutual dependence ... is too strong to be broke.”
—Laurence Sterne (1713–1768)
“Knowledge is of two kinds. We know a subject ourselves, or we know where we can find information upon it.”
—Samuel Johnson (1709–1784)