Idempotent Matrix - Applications

Applications

Idempotent matrices arise frequently in regression analysis and econometrics. For example, in ordinary least squares, the regression problem is to choose a vector of coefficient estimates so as to minimize the sum of squared residuals (mispredictions) e_i: in matrix form,

where y is a vector of dependent variable observations, and X is a matrix each of whose columns is a column of observations on one of the independent variables. The resulting estimator is

where superscript T indicates a transpose, and the vector of residuals is

Here both M and (Hat matrix) are idempotent matrices, a fact which allows simplification when the sum of squared residuals is computed:

The idempotency of M plays a role in other calculations as well, such as in determining the variance of the estimator .

An idempotent linear operator P is a projection operator on the range space R(P) along its null space N(P). P is an orthogonal projection operator if and only if it is idempotent and symmetric.