Method
- Divide the available data into training and validation sets.
- Use a large number of hidden units.
- Use very small random initial values.
- Use a slow learning rate.
- Compute the validation error rate periodically during training.
- Stop training when the validation error rate "starts to go up".
It is crucial to realize that the validation error is not a good estimate of the generalization error. One method for getting an unbiased estimate of the generalization error is to run the net on a third set of data, the test set, that is not used at all during the training process. The error on the test set gives estimate on generalization; to have the outputs of the net approximate target values given inputs that are not in the training set.
Read more about this topic: Early Stopping
Famous quotes containing the word method:
“The most passionate, consistent, extreme and implacable enemy of the Enlightenment and ... all forms of rationalism ... was Johann Georg Hamann. His influence, direct and indirect, upon the romantic revolt against universalism and scientific method ... was considerable and perhaps crucial.”
—Isaiah Berlin (b. 1909)
““I have usually found that there was method in his madness.”
“Some folk might say there was madness in his method.””
—Sir Arthur Conan Doyle (1859–1930)
“The country needs and, unless I mistake its temper, the country demands bold, persistent experimentation. It is common sense to take a method and try it. If it fails, admit it frankly and try another. But above all, try something. The millions who are in want will not stand idly by silently forever while the things to satisfy their needs are within easy reach.”
—Franklin D. Roosevelt (1882–1945)