Empirical Risk Minimization
In general, the risk cannot be computed because the distribution is unknown to the learning algorithm (this situation is referred to as agnostic learning). However, we can compute an approximation, called empirical risk, by averaging the loss function on the training set:
Empirical risk minimization principle states that the learning algorithm should choose a hypothesis which minimizes the empirical risk:
Thus the learning algorithm defined by the ERM principle consists in solving the above optimization problem.
Read more about this topic: Empirical Risk Minimization
Famous quotes containing the words empirical and/or risk:
“To develop an empiricist account of science is to depict it as involving a search for truth only about the empirical world, about what is actual and observable.... It must involve throughout a resolute rejection of the demand for an explanation of the regularities in the observable course of nature, by means of truths concerning a reality beyond what is actual and observable, as a demand which plays no role in the scientific enterprise.”
—Bas Van Fraassen (b. 1941)
“Better risk loss of truth than chance of error—that is your faith-vetoer’s exact position. He is actively playing his stake as much as the believer is; he is backing the field against the religious hypothesis, just as the believer is backing the religious hypothesis against the field.”
—William James (1842–1910)