Method
- Divide the available data into training and validation sets.
- Use a large number of hidden units.
- Use very small random initial values.
- Use a slow learning rate.
- Compute the validation error rate periodically during training.
- Stop training when the validation error rate "starts to go up".
It is crucial to realize that the validation error is not a good estimate of the generalization error. One method for getting an unbiased estimate of the generalization error is to run the net on a third set of data, the test set, that is not used at all during the training process. The error on the test set gives estimate on generalization; to have the outputs of the net approximate target values given inputs that are not in the training set.
Read more about this topic: Early Stopping
Famous quotes containing the word method:
“If all feeling for grace and beauty were not extinguished in the mass of mankind at the actual moment, such a method of locomotion as cycling could never have found acceptance; no man or woman with the slightest aesthetic sense could assume the ludicrous position necessary for it.”
—Ouida [Marie Louise De La Ramée] (1839–1908)
“As a science of the unconscious it is a therapeutic method, in the grand style, a method overarching the individual case. Call this, if you choose, a poet’s utopia.”
—Thomas Mann (1875–1955)
“We have not given science too big a place in our education, but we have made a perilous mistake in giving it too great a preponderance in method in every other branch of study.”
—Woodrow Wilson (1856–1924)