Tikhonov Regularization - Determination of The Tikhonov Factor

Determination of The Tikhonov Factor

The optimal regularization parameter is usually unknown and often in practical problems is determined by an ad hoc method. A possible approach relies on the Bayesian interpretation described above. Other approaches include the discrepancy principle, cross-validation, L-curve method, restricted maximum likelihood and unbiased predictive risk estimator. Grace Wahba proved that the optimal parameter, in the sense of leave-one-out cross-validation minimizes:

where is the residual sum of squares and is the effective number of degrees of freedom.

Using the previous SVD decomposition, we can simplify the above expression:

and

$\tau = m - \sum _{i=1} ^q \frac{\sigma _i ^2}{\sigma _i ^2 + \alpha ^2} = m - q + \sum _{i=1} ^q \frac{\alpha ^2}{\sigma _i ^2 + \alpha ^2}$

Read more about this topic: Tikhonov Regularization

Famous quotes containing the words determination of and/or factor:

“No man is quite sane; each has a vein of folly in his composition, a slight determination of blood to the head, to make sure of holding him hard to some one point which nature has taken to heart.”
—Ralph Waldo Emerson (1803–1882)

“Weapons are an important factor in war, but not the decisive factor; it is people, not things, that are decisive. The contest of strength is not only a contest of military and economic power, but also a contest of human power and morale. Military and economic power is necessarily wielded by people.”
—Mao Zedong (1893–1976)