Vector Quantization - Training

Training

A simple training algorithm for vector quantization is:

  1. Pick a sample point at random
  2. Move the nearest quantization vector centroid towards this sample point, by a small fraction of the distance
  3. Repeat

A more sophisticated algorithm reduces the bias in the density matching estimation, and ensures that all points are used, by including an extra sensitivity parameter:

  1. Increase each centroid's sensitivity by a small amount
  2. Pick a sample point at random
  3. Find the quantization vector centroid with the smallest
    1. Move the chosen centroid toward the sample point by a small fraction of the distance
    2. Set the chosen centroid's sensitivity to zero
  4. Repeat

It is desirable to use a cooling schedule to produce convergence: see Simulated annealing.

The algorithm can be iteratively updated with 'live' data, rather than by picking random points from a data set, but this will introduce some bias if the data is temporally correlated over many samples. A vector is represented either geometrically by an arrow whose length corresponds to its magnitude and points in an appropriate direction, or by two or three numbers representing the magnitude of its components.

Read more about this topic:  Vector Quantization

Famous quotes containing the word training:

    “I’m not suggesting that all men are beautiful, vulnerable boys, but we all started out that way. What happened to us? How did we become monsters of feminist nightmares? The answer, of course, is that we underwent a careful and deliberate process of gender training, sometimes brutal, always dehumanizing, cutting away large chunks of ourselves. Little girls went through something similarly crippling. If the gender training was successful, we each ended up being half a person.”
    —Frank Pittman (20th century)

    “When the child is twelve, your wife buys her a splendidly silly article of clothing called a training bra. To train what? I never had a training jock. And believe me, when I played football, I could have used a training jock more than any twelve-year-old needs a training bra.”
    —Bill Cosby (20th century)

    “There is all the difference in the world between departure from recognised rules by one who has learned to obey them, and neglect of them through want of training or want of skill or want of understanding. Before you can be eccentric you must know where the circle is.”
    —Ellen Terry (1847–1928)