Empirical Risk Minimization
In general, the risk cannot be computed because the distribution is unknown to the learning algorithm (this situation is referred to as agnostic learning). However, we can compute an approximation, called empirical risk, by averaging the loss function on the training set:
Empirical risk minimization principle states that the learning algorithm should choose a hypothesis which minimizes the empirical risk:
Thus the learning algorithm defined by the ERM principle consists in solving the above optimization problem.
Read more about this topic: Empirical Risk Minimization
Famous quotes containing the words empirical and/or risk:
“To develop an empiricist account of science is to depict it as involving a search for truth only about the empirical world, about what is actual and observable.... It must involve throughout a resolute rejection of the demand for an explanation of the regularities in the observable course of nature, by means of truths concerning a reality beyond what is actual and observable, as a demand which plays no role in the scientific enterprise.”
—Bas Van Fraassen (b. 1941)
“Nature, we are starting to realize, is every bit as important as nurture. Genetic influences, brain chemistry, and neurological development contribute strongly to who we are as children and what we become as adults. For example, tendencies to excessive worrying or timidity, leadership qualities, risk taking, obedience to authority, all appear to have a constitutional aspect.”
—Stanley Turecki (20th century)