Eigendecomposition of A Matrix - Fundamental Theory of Matrix Eigenvectors and Eigenvalues

Fundamental Theory of Matrix Eigenvectors and Eigenvalues

A (non-zero) vector v of dimension N is an eigenvector of a square (N×N) matrix A if and only if it satisfies the linear equation

where λ is a scalar, termed the eigenvalue corresponding to v. That is, the eigenvectors are the vectors which the linear transformation A merely elongates or shrinks, and the amount that they elongate/shrink by is the eigenvalue. The above equation is called the eigenvalue equation or the eigenvalue problem.

This yields an equation for the eigenvalues

We call p(λ) the characteristic polynomial, and the equation, called the characteristic equation, is an Nth order polynomial equation in the unknown λ. This equation will have N_λ distinct solutions, where 1 ≤ N_λ ≤ N . The set of solutions, i.e. the eigenvalues, is sometimes called the spectrum of A.

We can factor p as

The integer n_i is termed the algebraic multiplicity of eigenvalue λ_i. The algebraic multiplicities sum to N:

For each eigenvalue, λ_i, we have a specific eigenvalue equation

There will be 1 ≤ m_i ≤ n_i linearly independent solutions to each eigenvalue equation. The m_i solutions are the eigenvectors associated with the eigenvalue λ_i. The integer m_i is termed the geometric multiplicity of λ_i. It is important to keep in mind that the algebraic multiplicity n_i and geometric multiplicity m_i may or may not be equal, but we always have m_i ≤ n_i. The simplest case is of course when m_i = n_i = 1. The total number of linearly independent eigenvectors, N_v, can be calculated by summing the geometric multiplicities

The eigenvectors can be indexed by eigenvalues, i.e. using a double index, with v_i,j being the jth eigenvector for the ith eigenvalue. The eigenvectors can also be indexed using the simpler notation of a single index v_k, with k = 1, 2, ..., N_v.