Method
- Divide the available data into training and validation sets.
- Use a large number of hidden units.
- Use very small random initial values.
- Use a slow learning rate.
- Compute the validation error rate periodically during training.
- Stop training when the validation error rate "starts to go up".
It is crucial to realize that the validation error is not a good estimate of the generalization error. One method for getting an unbiased estimate of the generalization error is to run the net on a third set of data, the test set, that is not used at all during the training process. The error on the test set gives estimate on generalization; to have the outputs of the net approximate target values given inputs that are not in the training set.
Read more about this topic: Early Stopping
Famous quotes containing the word method:
“It is to be lamented that the principle of national has had very little nourishment in our country, and, instead, has given place to sectional or state partialities. What more promising method for remedying this defect than by uniting American women of every state and every section in a common effort for our whole country.”
—Catherine E. Beecher (1800–1878)
“Direct action ... is the logical, consistent method of Anarchism.”
—Emma Goldman (1869–1940)
“Letters are above all useful as a means of expressing the ideal self; and no other method of communication is quite so good for this purpose.... In letters we can reform without practice, beg without humiliation, snip and shape embarrassing experiences to the measure of our own desires....”
—Elizabeth Hardwick (b. 1916)