Formal Description
Take to be the vector space of all possible inputs, and to be the vector space of all possible outputs. Statistical learning theory takes the perspective that there is some unknown probability distribution over the product space, i.e. there exists some unknown . The training set is made up of samples from this probability distribution, and is notated
Every is an input vector from the training data, and is the output that corresponds to it.
In this formalism, the inference problem consists of finding a function such that . Let be a space of functions called the hypothesis space. The hypothesis space is the space of functions the algorithm will search through. Let be the ], a metric for the difference between the predicted value and the actual value . The expected risk is defined to be
The target function, the best possible function that can be chosen, is given by the that satisfies
Because the probability distribution is unknown, a proxy measure for the expected risk must be used. This measure is based on the training set, a sample from this unknown probability distribution. It is called the empirical risk
A learning algorithm that chooses the function that minimizes the empirical risk is called empirical risk minimization.
Read more about this topic: Statistical Learning Theory
Famous quotes containing the words formal and/or description:
“The spiritual kinship between Lincoln and Whitman was founded upon their Americanism, their essential Westernism. Whitman had grown up without much formal education; Lincoln had scarcely any education. One had become the notable poet of the day; one the orator of the Gettsyburg Address. It was inevitable that Whitman as a poet should turn with a feeling of kinship to Lincoln, and even without any association or contact feel that Lincoln was his.”
—Edgar Lee Masters (1869–1950)
“An intentional object is given by a word or a phrase which gives a description under which.”
—Gertrude Elizabeth Margaret Anscombe (b. 1919)