Conjugate Gradient On The Normal Equations
The conjugate gradient method can be applied to an arbitrary n-by-m matrix by applying it to normal equations ATA and right-hand side vector ATb, since ATA is a symmetric positive-semidefinite matrix for any A. The result is conjugate gradient on the normal equations (CGNR).
- ATAx = ATb
As an iterative method, it is not necessary to form ATA explicitly in memory but only to perform the matrix-vector and transpose matrix-vector multiplications. Therefore CGNR is particularly useful when A is a sparse matrix since these operations are usually extremely efficient. However the downside of forming the normal equations is that the condition number κ(ATA) is equal to κ2(A) and so the rate of convergence of CGNR may be slow and the quality of the approximate solution may be sensitive to roundoff errors. Finding a good preconditioner is often an important part of using the CGNR method.
Several algorithms have been proposed (e.g., CGLS, LSQR). The LSQR algorithm purportedly has the best numerical stability when A is ill-conditioned, i.e., A has a large condition number.
Read more about this topic: Conjugate Gradient Method
Famous quotes containing the word normal:
“You know that fiction, prose rather, is possibly the roughest trade of all in writing. You do not have the reference, the old important reference. You have the sheet of blank paper, the pencil, and the obligation to invent truer than things can be true. You have to take what is not palpable and make it completely palpable and also have it seem normal and so that it can become a part of experience of the person who reads it.”
—Ernest Hemingway (1899–1961)