Empirical Risk Minimization
In general, the risk cannot be computed because the distribution is unknown to the learning algorithm (this situation is referred to as agnostic learning). However, we can compute an approximation, called empirical risk, by averaging the loss function on the training set:
Empirical risk minimization principle states that the learning algorithm should choose a hypothesis which minimizes the empirical risk:
Thus the learning algorithm defined by the ERM principle consists in solving the above optimization problem.
Read more about this topic: Empirical Risk Minimization
Famous quotes containing the words empirical and/or risk:
“To develop an empiricist account of science is to depict it as involving a search for truth only about the empirical world, about what is actual and observable.... It must involve throughout a resolute rejection of the demand for an explanation of the regularities in the observable course of nature, by means of truths concerning a reality beyond what is actual and observable, as a demand which plays no role in the scientific enterprise.”
—Bas Van Fraassen (b. 1941)
“If you think it will only add one sprig to the wreath the country twines to bind the brows of my hero, I will run the risk of being sneered at by those who criticize female productions of all kinds. ...Though a female, I was born a patriot.”
—Annie Boudinot Stockton (1736–1801)