Tikhonov Regularization - Determination of The Tikhonov Factor

Determination of The Tikhonov Factor

The optimal regularization parameter is usually unknown and often in practical problems is determined by an ad hoc method. A possible approach relies on the Bayesian interpretation described above. Other approaches include the discrepancy principle, cross-validation, L-curve method, restricted maximum likelihood and unbiased predictive risk estimator. Grace Wahba proved that the optimal parameter, in the sense of leave-one-out cross-validation minimizes:

where is the residual sum of squares and is the effective number of degrees of freedom.

Using the previous SVD decomposition, we can simplify the above expression:

and

$\tau = m - \sum _{i=1} ^q \frac{\sigma _i ^2}{\sigma _i ^2 + \alpha ^2} = m - q + \sum _{i=1} ^q \frac{\alpha ^2}{\sigma _i ^2 + \alpha ^2}$

Read more about this topic: Tikhonov Regularization

Famous quotes containing the words determination of and/or factor:

“War should be carried on like a monsoon; one changeless determination of every particle towards the one unalterable aim.”
—Herman Melville (1819–1891)

“Children of the middle years do not do their learning unaffected by attendant feelings of interest, boredom, success, failure, chagrin, joy, humiliation, pleasure, distress and delight. They are whole children responding in a total way, and what they feel is a constant factor that can be constructive or destructive in any learning situation.”
—Dorothy H. Cohen (20th century)