Principal Component Analysis - Derivation of PCA Using The Covariance Method

Derivation of PCA Using The Covariance Method

Let X be a d-dimensional random vector expressed as column vector. Without loss of generality, assume X has zero mean.

We want to find a orthonormal transformation matrix P so that PX has a diagonal covariant matrix (i.e. PX is a random vector with all its distinct components pairwise uncorrelated).

A quick computation assuming were unitary yields:

\begin{array}{rcl} \operatorname{var}(PX) &= &\mathbb{E}\\ &= &\mathbb{E}\\ &= &P~\mathbb{E}P^{\dagger}\\ &= &P~\operatorname{cov}(X)P^{-1}\\ \end{array}

Hence holds if and only if were diagonalisable by .

This is very constructive, as var(X) is guaranteed to be a non-negative definite matrix and thus is guaranteed to be diagonalisable by some unitary matrix.

Read more about this topic:  Principal Component Analysis

Famous quotes containing the word method:

    “Letters are above all useful as a means of expressing the ideal self; and no other method of communication is quite so good for this purpose.... In letters we can reform without practice, beg without humiliation, snip and shape embarrassing experiences to the measure of our own desires....”
    —Elizabeth Hardwick (b. 1916)