Formal Description
Take to be the vector space of all possible inputs, and to be the vector space of all possible outputs. Statistical learning theory takes the perspective that there is some unknown probability distribution over the product space, i.e. there exists some unknown . The training set is made up of samples from this probability distribution, and is notated
Every is an input vector from the training data, and is the output that corresponds to it.
In this formalism, the inference problem consists of finding a function such that . Let be a space of functions called the hypothesis space. The hypothesis space is the space of functions the algorithm will search through. Let be the ], a metric for the difference between the predicted value and the actual value . The expected risk is defined to be
The target function, the best possible function that can be chosen, is given by the that satisfies
Because the probability distribution is unknown, a proxy measure for the expected risk must be used. This measure is based on the training set, a sample from this unknown probability distribution. It is called the empirical risk
A learning algorithm that chooses the function that minimizes the empirical risk is called empirical risk minimization.
Read more about this topic: Statistical Learning Theory
Famous quotes containing the words formal and/or description:
“This is no argument against teaching manners to the young. On the contrary, it is a fine old tradition that ought to be resurrected from its current mothballs and put to work...In fact, children are much more comfortable when they know the guide rules for handling the social amenities. It’s no more fun for a child to be introduced to a strange adult and have no idea what to say or do than it is for a grownup to go to a formal dinner and have no idea what fork to use.”
—Leontine Young (20th century)
“The next Augustan age will dawn on the other side of the Atlantic. There will, perhaps, be a Thucydides at Boston, a Xenophon at New York, and, in time, a Virgil at Mexico, and a Newton at Peru. At last, some curious traveller from Lima will visit England and give a description of the ruins of St Paul’s, like the editions of Balbec and Palmyra.”
—Horace Walpole (1717–1797)