Conjugate Gradient On The Normal Equations
The conjugate gradient method can be applied to an arbitrary n-by-m matrix by applying it to normal equations ATA and right-hand side vector ATb, since ATA is a symmetric positive-semidefinite matrix for any A. The result is conjugate gradient on the normal equations (CGNR).
- ATAx = ATb
As an iterative method, it is not necessary to form ATA explicitly in memory but only to perform the matrix-vector and transpose matrix-vector multiplications. Therefore CGNR is particularly useful when A is a sparse matrix since these operations are usually extremely efficient. However the downside of forming the normal equations is that the condition number κ(ATA) is equal to κ2(A) and so the rate of convergence of CGNR may be slow and the quality of the approximate solution may be sensitive to roundoff errors. Finding a good preconditioner is often an important part of using the CGNR method.
Several algorithms have been proposed (e.g., CGLS, LSQR). The LSQR algorithm purportedly has the best numerical stability when A is ill-conditioned, i.e., A has a large condition number.
Read more about this topic: Conjugate Gradient Method
Famous quotes containing the word normal:
“As blacks, we need not be afraid that encouraging moral development, a conscience and guilt will prevent social action. Black children without the ability to feel a normal amount of guilt will victimize their parents, relatives and community first. They are unlikely to be involved in social action to improve the black community. Their self-centered personalities will cause them to look out for themselves without concern for others, black or white.”
—James P. Comer (20th century)