Method
- Divide the available data into training and validation sets.
- Use a large number of hidden units.
- Use very small random initial values.
- Use a slow learning rate.
- Compute the validation error rate periodically during training.
- Stop training when the validation error rate "starts to go up".
It is crucial to realize that the validation error is not a good estimate of the generalization error. One method for getting an unbiased estimate of the generalization error is to run the net on a third set of data, the test set, that is not used at all during the training process. The error on the test set gives estimate on generalization; to have the outputs of the net approximate target values given inputs that are not in the training set.
Read more about this topic: Early Stopping
Famous quotes containing the word method:
“Women stand related to beautiful nature around us, and the enamoured youth mixes their form with moon and stars, with woods and waters, and the pomp of summer. They heal us of awkwardness by their words and looks. We observe their intellectual influence on the most serious student. They refine and clear his mind: teach him to put a pleasing method into what is dry and difficult.”
—Ralph Waldo Emerson (1803–1882)
“The method of authority will always govern the mass of mankind; and those who wield the various forms of organized force in the state will never be convinced that dangerous reasoning ought not to be suppressed in some way.”
—Charles Sanders Peirce (1839–1914)
“There is assuredly no more effectual method of clearing up one’s own mind on any subject than by talking it over, so to speak, with men of real power and grasp, who have considered it from a totally different point of view.”
—Thomas Henry Huxley (1825–95)