Principal Component Analysis - Derivation of PCA Using The Covariance Method

Derivation of PCA Using The Covariance Method

Let X be a d-dimensional random vector expressed as column vector. Without loss of generality, assume X has zero mean.

We want to find a orthonormal transformation matrix P so that PX has a diagonal covariant matrix (i.e. PX is a random vector with all its distinct components pairwise uncorrelated).

A quick computation assuming were unitary yields:

\begin{array}{rcl} \operatorname{var}(PX) &= &\mathbb{E}\\ &= &\mathbb{E}\\ &= &P~\mathbb{E}P^{\dagger}\\ &= &P~\operatorname{cov}(X)P^{-1}\\ \end{array}

Hence holds if and only if were diagonalisable by .

This is very constructive, as var(X) is guaranteed to be a non-negative definite matrix and thus is guaranteed to be diagonalisable by some unitary matrix.

Read more about this topic:  Principal Component Analysis

Famous quotes containing the word method:

    “Traditional scientific method has always been at the very best 20-20 hindsight. It’s good for seeing where you’ve been. It’s good for testing the truth of what you think you know, but it can’t tell you where you ought to go.”
    —Robert M. Pirsig (b. 1928)