Tikhonov Regularization - Determination of The Tikhonov Factor

Determination of The Tikhonov Factor

The optimal regularization parameter is usually unknown and often in practical problems is determined by an ad hoc method. A possible approach relies on the Bayesian interpretation described above. Other approaches include the discrepancy principle, cross-validation, L-curve method, restricted maximum likelihood and unbiased predictive risk estimator. Grace Wahba proved that the optimal parameter, in the sense of leave-one-out cross-validation minimizes:

where is the residual sum of squares and is the effective number of degrees of freedom.

Using the previous SVD decomposition, we can simplify the above expression:

and

$\tau = m - \sum _{i=1} ^q \frac{\sigma _i ^2}{\sigma _i ^2 + \alpha ^2} = m - q + \sum _{i=1} ^q \frac{\alpha ^2}{\sigma _i ^2 + \alpha ^2}$

Read more about this topic: Tikhonov Regularization

Famous quotes containing the words determination of and/or factor:

“If an opinion can eventually go to the determination of a practical belief, it, in so far, becomes itself a practical belief; and every proposition that is not pure metaphysical jargon and chatter must have some possible bearing upon practice.”
—Charles Sanders Peirce (1839–1914)

“It is change, continuing change, inevitable change, that is the dominant factor in society today. No sensible decision can be made any longer without taking into account not only the world as it is, but the world as it will be.... This, in turn, means that our statesmen, our businessmen, our everyman must take on a science fictional way of thinking.”
—Isaac Asimov (1920–1992)