Tikhonov Regularization - Determination of The Tikhonov Factor

Determination of The Tikhonov Factor

The optimal regularization parameter is usually unknown and often in practical problems is determined by an ad hoc method. A possible approach relies on the Bayesian interpretation described above. Other approaches include the discrepancy principle, cross-validation, L-curve method, restricted maximum likelihood and unbiased predictive risk estimator. Grace Wahba proved that the optimal parameter, in the sense of leave-one-out cross-validation minimizes:

where is the residual sum of squares and is the effective number of degrees of freedom.

Using the previous SVD decomposition, we can simplify the above expression:

and

$\tau = m - \sum _{i=1} ^q \frac{\sigma _i ^2}{\sigma _i ^2 + \alpha ^2} = m - q + \sum _{i=1} ^q \frac{\alpha ^2}{\sigma _i ^2 + \alpha ^2}$

Read more about this topic: Tikhonov Regularization

Famous quotes containing the word factor:

“Weapons are an important factor in war, but not the decisive factor; it is people, not things, that are decisive. The contest of strength is not only a contest of military and economic power, but also a contest of human power and morale. Military and economic power is necessarily wielded by people.”
—Mao Zedong (1893–1976)