Relation To Mutual Information
For analysis of contingency tables the value of G can also be expressed in terms of mutual information.
Let
- , and
Then G can be expressed in several alternative forms:
where the entropy of a discrete random variable is defined as
and where
is the mutual information between the row vector and the column vector of the contingency table.
It can also be shown that the inverse document frequency weighting commonly used for text retrieval is an approximation of G applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the G statistic.
Read more about this topic: G-test
Famous quotes containing the words relation to, relation, mutual and/or information:
“The adolescent does not develop her identity and individuality by moving outside her family. She is not triggered by some magic unconscious dynamic whereby she rejects her family in favour of her peers or of a larger society.... She continues to develop in relation to her parents. Her mother continues to have more influence over her than either her father or her friends.”
—Terri Apter (20th century)
“Unaware of the absurdity of it, we introduce our own petty household rules into the economy of the universe for which the life of generations, peoples, of entire planets, has no importance in relation to the general development.”
—Alexander Herzen (1812–1870)
“Ties of blood are not always ties of friendship; but friendship founded on merit, on esteem, and on mutual trust, becomes more vital and more tender when strengthened by the ties of blood.”
—Philip Dormer Stanhope, 4th Earl Chesterfield (1694–1773)
“Phenomenal nature shadows him wherever he goes. Clouds in the staring sky transmit to one another, by means of slow signs, incredibly detailed information regarding him. His inmost thoughts are discussed at nightfall, in manual alphabet, by darkly gesticulating trees. Pebbles or stains or sunflecks form patterns representing in some awful way messages which he must intercept. Everything is a cipher and of everything he is the theme.”
—Vladimir Nabokov (1899–1977)