Formal Description
Take to be the vector space of all possible inputs, and to be the vector space of all possible outputs. Statistical learning theory takes the perspective that there is some unknown probability distribution over the product space, i.e. there exists some unknown . The training set is made up of samples from this probability distribution, and is notated
Every is an input vector from the training data, and is the output that corresponds to it.
In this formalism, the inference problem consists of finding a function such that . Let be a space of functions called the hypothesis space. The hypothesis space is the space of functions the algorithm will search through. Let be the ], a metric for the difference between the predicted value and the actual value . The expected risk is defined to be
The target function, the best possible function that can be chosen, is given by the that satisfies
Because the probability distribution is unknown, a proxy measure for the expected risk must be used. This measure is based on the training set, a sample from this unknown probability distribution. It is called the empirical risk
A learning algorithm that chooses the function that minimizes the empirical risk is called empirical risk minimization.
Read more about this topic: Statistical Learning Theory
Famous quotes containing the words formal and/or description:
“It is in the nature of allegory, as opposed to symbolism, to beg the question of absolute reality. The allegorist avails himself of a formal correspondence between “ideas” and “things,” both of which he assumes as given; he need not inquire whether either sphere is “real” or whether, in the final analysis, reality consists in their interaction.”
—Charles, Jr. Feidelson, U.S. educator, critic. Symbolism and American Literature, ch. 1, University of Chicago Press (1953)
“A sound mind in a sound body, is a short, but full description of a happy state in this World: he that has these two, has little more to wish for; and he that wants either of them, will be little the better for anything else.”
—John Locke (1632–1704)