The Square of The Correlation and Explained (or Informational) Variation
In cases where (Y, X) are such that the conditional expected value is linear; i.e., in cases where
it follows from the bilinearity of Cov(-,-) that
and
and the explained component of the variance divided by the total variance is just the square of the correlation between Y and X; i.e., in such cases,
One example of this situation is when (X, Y) have a bivariate normal (Gaussian) distribution.
More generally, when the conditional expectation E( Y | X ) is a non-linear function of X
which can be estimated as the R squared from a non-linear regression of Y on X, using data drawn from the joint distribution of (X,Y). When E( Y | X ) has a Gaussian distribution (and is an invertible function of X), or Y itself has a (marginal) Gaussian distribution, this explained component of variation sets a lower bound on the mutual information:
Read more about this topic: Law Of Total Variance
Famous quotes containing the words square and/or explained:
“If magistrates had true justice, and if physicians had the true art of healing, they would have no occasion for square caps; the majesty of these sciences would of itself be venerable enough. But having only imaginary knowledge, they must employ those silly tools that strike the imagination with which they have to deal; and thereby, in fact, they inspire respect.”
—Blaise Pascal (1623–1662)
“It is explained that all relationships require a little give and take. This is untrue. Any partnership demands that we give and give and give and at the last, as we flop into our graves exhausted, we are told that we didn’t give enough.”
—Quentin Crisp (b. 1908)