Vector Quantization - Training

Training

A simple training algorithm for vector quantization is:

  1. Pick a sample point at random
  2. Move the nearest quantization vector centroid towards this sample point, by a small fraction of the distance
  3. Repeat

A more sophisticated algorithm reduces the bias in the density matching estimation, and ensures that all points are used, by including an extra sensitivity parameter:

  1. Increase each centroid's sensitivity by a small amount
  2. Pick a sample point at random
  3. Find the quantization vector centroid with the smallest
    1. Move the chosen centroid toward the sample point by a small fraction of the distance
    2. Set the chosen centroid's sensitivity to zero
  4. Repeat

It is desirable to use a cooling schedule to produce convergence: see Simulated annealing.

The algorithm can be iteratively updated with 'live' data, rather than by picking random points from a data set, but this will introduce some bias if the data is temporally correlated over many samples. A vector is represented either geometrically by an arrow whose length corresponds to its magnitude and points in an appropriate direction, or by two or three numbers representing the magnitude of its components.

Read more about this topic:  Vector Quantization

Famous quotes containing the word training:

    “The area [of toilet training] is one where a child really does possess the power to defy. Strong pressure leads to a powerful struggle. The issue then is not toilet training but who holds the reins—mother or child? And the child has most of the ammunition!”
    —Dorothy Corkville Briggs (20th century)

    “The triumphs of peace have been in some proximity to war. Whilst the hand was still familiar with the sword-hilt, whilst the habits of the camp were still visible in the port and complexion of the gentleman, his intellectual power culminated; the compression and tension of these stern conditions is a training for the finest and softest arts, and can rarely be compensated in tranquil times, except by some analogous vigor drawn from occupations as hardy as war.”
    —Ralph Waldo Emerson (1803–1882)

    “Perhaps the most valuable result of all education is the ability to make yourself do the thing you have to do, when it ought to be done, whether you like it or not; it is the first lesson that ought to be learned; and however early a man’s training begins, its probably the last lesson that he learns thoroughly.”
    —Thomas Henry Huxley (1825–95)