Multivariate Normal Distribution - Conditional Distributions

Conditional Distributions

If μ and Σ are partitioned as follows


\boldsymbol\mu
=
\begin{bmatrix} \boldsymbol\mu_1 \\ \boldsymbol\mu_2
\end{bmatrix}
\quad with sizes

\boldsymbol\Sigma
=
\begin{bmatrix} \boldsymbol\Sigma_{11} & \boldsymbol\Sigma_{12} \\ \boldsymbol\Sigma_{21} & \boldsymbol\Sigma_{22}
\end{bmatrix}
\quad with sizes

then, the distribution of x1 conditional on x2 = a is multivariate normal (x1|x2 = a) ~ N(μ, Σ) where


\bar{\boldsymbol\mu}
=
\boldsymbol\mu_1 + \boldsymbol\Sigma_{12} \boldsymbol\Sigma_{22}^{-1}
\left( \mathbf{a} - \boldsymbol\mu_2
\right)

and covariance matrix


\overline{\boldsymbol\Sigma}
=
\boldsymbol\Sigma_{11} - \boldsymbol\Sigma_{12} \boldsymbol\Sigma_{22}^{-1} \boldsymbol\Sigma_{21}.

This matrix is the Schur complement of Σ22 in Σ. This means that to calculate the conditional covariance matrix, one inverts the overall covariance matrix, drops the rows and columns corresponding to the variables being conditioned upon, and then inverts back to get the conditional covariance matrix. Here is the generalized inverse of

Note that knowing that x2 = a alters the variance, though the new variance does not depend on the specific value of a; perhaps more surprisingly, the mean is shifted by ; compare this with the situation of not knowing the value of a, in which case x1 would have distribution .

An interesting fact derived in order to prove this result, is that the random vectors and are independent.

The matrix Σ12Σ22−1 is known as the matrix of regression coefficients.

In the bivariate case where x is partitioned into X1 and X2, the conditional distribution of X1 given X2 is

where is the correlation coefficient between X1 and X2.

Read more about this topic:  Multivariate Normal Distribution

Famous quotes containing the word conditional:

    Conditional love is love that is turned off and on....Some parents only show their love after a child has done something that pleases them. “I love you, honey, for cleaning your room!” Children who think they need to earn love become people pleasers, or perfectionists. Those who are raised on conditional love never really feel loved.
    Louise Hart (20th century)