Relation To Mutual Information
For analysis of contingency tables the value of G can also be expressed in terms of mutual information.
Let
- , and
Then G can be expressed in several alternative forms:
where the entropy of a discrete random variable is defined as
and where
is the mutual information between the row vector and the column vector of the contingency table.
It can also be shown that the inverse document frequency weighting commonly used for text retrieval is an approximation of G applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the G statistic.
Read more about this topic: G-test
Famous quotes containing the words relation to, relation, mutual and/or information:
“Science is the language of the temporal world; love is that of the spiritual world. Man, indeed, describes more than he explains; while the angelic spirit sees and understands. Science saddens man; love enraptures the angel; science is still seeking, love has found. Man judges of nature in relation to itself; the angelic spirit judges of it in relation to heaven. In short to the spirits everything speaks.”
—Honoré De Balzac (1799–1850)
“There is a relation between the hours of our life and the centuries of time. As the air I breathe is drawn from the great repositories of nature, as the light on my book is yielded by a star a hundred millions of miles distant, as the poise of my body depends on the equilibrium of centrifugal and centripetal forces, so the hours should be instructed by the ages and the ages explained by the hours.”
—Ralph Waldo Emerson (1803–1882)
“Rules and particular inferences alike are justified by being brought into agreement with each other. A rule is amended if it yields an inference we are unwilling to accept; an inference is rejected if it violates a rule we are unwilling to amend. The process of justification is the delicate one of making mutual adjustments between rules and accepted inferences; and in the agreement achieved lies the only justification needed for either.”
—Nelson Goodman (b. 1906)
“English literature is a kind of training in social ethics.... English trains you to handle a body of information in a way that is conducive to action.”
—Marilyn Butler (b. 1937)