Vector Quantization - Training

Training

A simple training algorithm for vector quantization is:

  1. Pick a sample point at random
  2. Move the nearest quantization vector centroid towards this sample point, by a small fraction of the distance
  3. Repeat

A more sophisticated algorithm reduces the bias in the density matching estimation, and ensures that all points are used, by including an extra sensitivity parameter:

  1. Increase each centroid's sensitivity by a small amount
  2. Pick a sample point at random
  3. Find the quantization vector centroid with the smallest
    1. Move the chosen centroid toward the sample point by a small fraction of the distance
    2. Set the chosen centroid's sensitivity to zero
  4. Repeat

It is desirable to use a cooling schedule to produce convergence: see Simulated annealing.

The algorithm can be iteratively updated with 'live' data, rather than by picking random points from a data set, but this will introduce some bias if the data is temporally correlated over many samples. A vector is represented either geometrically by an arrow whose length corresponds to its magnitude and points in an appropriate direction, or by two or three numbers representing the magnitude of its components.

Read more about this topic:  Vector Quantization

Famous quotes containing the word training:

    “Unfortunately, life may sometimes seem unfair to middle children, some of whom feel like an afterthought to a brilliant older sibling and unable to captivate the family’s attention like the darling baby. Yet the middle position offers great training for the real world of lowered expectations, negotiation, and compromise. Middle children who often must break the mold set by an older sibling may thereby learn to challenge family values and seek their own identity.”
    —Marianne E. Neifert (20th century)

    “I am not a suffragist, nor do I believe in “careers” for women, especially a “career” in factory and mill where most working women have their “careers.” A great responsibility rests upon woman—the training of children. This is her most beautiful task.”
    —Mother Jones (1830–1930)

    “When a man goes through six years’ training to be a doctor he will never be the same. He knows too much.”
    —Enid Bagnold (1889–1981)