Law of Total Variance - The Square of The Correlation and Explained (or Informational) Variation

The Square of The Correlation and Explained (or Informational) Variation

In cases where (Y, X) are such that the conditional expected value is linear; i.e., in cases where

it follows from the bilinearity of Cov(-,-) that

and

and the explained component of the variance divided by the total variance is just the square of the correlation between Y and X; i.e., in such cases,

One example of this situation is when (X, Y) have a bivariate normal (Gaussian) distribution.

More generally, when the conditional expectation E( Y | X ) is a non-linear function of X

which can be estimated as the R squared from a non-linear regression of Y on X, using data drawn from the joint distribution of (X,Y). When E( Y | X ) has a Gaussian distribution (and is an invertible function of X), or Y itself has a (marginal) Gaussian distribution, this explained component of variation sets a lower bound on the mutual information: