Formal Description
Take to be the vector space of all possible inputs, and to be the vector space of all possible outputs. Statistical learning theory takes the perspective that there is some unknown probability distribution over the product space, i.e. there exists some unknown . The training set is made up of samples from this probability distribution, and is notated
Every is an input vector from the training data, and is the output that corresponds to it.
In this formalism, the inference problem consists of finding a function such that . Let be a space of functions called the hypothesis space. The hypothesis space is the space of functions the algorithm will search through. Let be the ], a metric for the difference between the predicted value and the actual value . The expected risk is defined to be
The target function, the best possible function that can be chosen, is given by the that satisfies
Because the probability distribution is unknown, a proxy measure for the expected risk must be used. This measure is based on the training set, a sample from this unknown probability distribution. It is called the empirical risk
A learning algorithm that chooses the function that minimizes the empirical risk is called empirical risk minimization.
Read more about this topic: Statistical Learning Theory
Famous quotes containing the words formal and/or description:
“True variety is in that plenitude of real and unexpected elements, in the branch charged with blue flowers thrusting itself, against all expectations, from the springtime hedge which seems already too full, while the purely formal imitation of variety ... is but void and uniformity, that is, that which is most opposed to variety....”
—Marcel Proust (1871–1922)
“I fancy it must be the quantity of animal food eaten by the English which renders their character insusceptible of civilisation. I suspect it is in their kitchens and not in their churches that their reformation must be worked, and that Missionaries of that description from [France] would avail more than those who should endeavor to tame them by precepts of religion or philosophy.”
—Thomas Jefferson (1743–1826)