Early Stopping

In machine learning, early stopping is a form of regularization used when a machine learning model (such as a neural network) is trained by on-line gradient descent. In early stopping, the training set is split into a new training set and a validation set. Gradient descent is applied to the new training set. After each sweep through the new training set, the network is evaluated on the validation set. When the performance with the validation test stops improving, the algorithm halts. The network with the best performance on the validation set is then used for actual testing, with a separate set of data (the validation set is used in learning to decide when to stop).

This technique is a simple but efficient hack to deal with the problem of overfitting. Overfitting is a phenomenon in which a learning system, such as a neural network gets very good at dealing with one data set at the expense of becoming very bad at dealing with other data sets. Early stopping is effectively limiting the used weights in the network and thus imposes a regularization, effectively lowering the VC dimension.

Early stopping is a very common practice in neural network training and often produces networks that generalize well. However, while often improving the generalization it does not do so in a mathematically well-defined way.

Read more about Early Stopping:  Method, Advantages, Issues

Famous quotes containing the words early and/or stopping:

    The shift from the perception of the child as innocent to the perception of the child as competent has greatly increased the demands on contemporary children for maturity, for participating in competitive sports, for early academic achievement, and for protecting themselves against adults who might do them harm. While children might be able to cope with any one of those demands taken singly, taken together they often exceed children’s adaptive capacity.
    David Elkind (20th century)

    There seems to be no stopping drug frenzy once it takes hold of a nation. What starts with an innocuous HUGS, NOT DRUGS bumper sticker soon leads to wild talk of shooting dealers and making urine tests a condition for employment—anywhere.
    Barbara Ehrenreich (b. 1941)