Relation To Mutual Information
For analysis of contingency tables the value of G can also be expressed in terms of mutual information.
Let
- , and
Then G can be expressed in several alternative forms:
where the entropy of a discrete random variable is defined as
and where
is the mutual information between the row vector and the column vector of the contingency table.
It can also be shown that the inverse document frequency weighting commonly used for text retrieval is an approximation of G applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the G statistic.
Read more about this topic: G-test
Famous quotes containing the words relation to, relation, mutual and/or information:
“The difference between objective and subjective extension is one of relation to a context solely.”
—William James (1842–1910)
“There is undoubtedly something religious about it: everyone believes that they are special, that they are chosen, that they have a special relation with fate. Here is the test: you turn over card after card to see in which way that is true. If you can defy the odds, you may be saved. And when you are cleaned out, the last penny gone, you are enlightened at last, free perhaps, exhilarated like an ascetic by the falling away of the material world.”
—Andrei Codrescu (b. 1947)
“Ties of blood are not always ties of friendship; but friendship founded on merit, on esteem, and on mutual trust, becomes more vital and more tender when strengthened by the ties of blood.”
—Philip Dormer Stanhope, 4th Earl Chesterfield (1694–1773)
“Information networks straddle the world. Nothing remains concealed. But the sheer volume of information dissolves the information. We are unable to take it all in.”
—Günther Grass (b. 1927)