Early Stopping - Method

Method

  1. Divide the available data into training and validation sets.
  2. Use a large number of hidden units.
  3. Use very small random initial values.
  4. Use a slow learning rate.
  5. Compute the validation error rate periodically during training.
  6. Stop training when the validation error rate "starts to go up".

It is crucial to realize that the validation error is not a good estimate of the generalization error. One method for getting an unbiased estimate of the generalization error is to run the net on a third set of data, the test set, that is not used at all during the training process. The error on the test set gives estimate on generalization; to have the outputs of the net approximate target values given inputs that are not in the training set.

Read more about this topic:  Early Stopping

Famous quotes containing the word method:

    “If all feeling for grace and beauty were not extinguished in the mass of mankind at the actual moment, such a method of locomotion as cycling could never have found acceptance; no man or woman with the slightest aesthetic sense could assume the ludicrous position necessary for it.”
    —Ouida [Marie Louise De La Ramée] (1839–1908)

    “Argument is conclusive ... but ... it does not remove doubt, so that the mind may rest in the sure knowledge of the truth, unless it finds it by the method of experiment.... For if any man who never saw fire proved by satisfactory arguments that fire burns ... his hearer’s mind would never be satisfied, nor would he avoid the fire until he put his hand in it ... that he might learn by experiment what argument taught.”
    —Roger Bacon (c. 1214–1294)

    “No method nor discipline can supersede the necessity of being forever on the alert. What is a course of history or philosophy, or poetry, no matter how well selected, or the best society, or the most admirable routine of life, compared with the discipline of looking always at what is to be seen? Will you be a reader, a student merely, or a seer? Read your fate, see what is before you, and walk on into futurity.”
    —Henry David Thoreau (1817–1862)