k-nearest Neighbor Algorithm - For Estimating Continuous Variables

For Estimating Continuous Variables

The k-NN algorithm can also be adapted for use in estimating continuous variables. One such implementation uses an inverse distance weighted average of the k-nearest multivariate neighbors. This algorithm functions as follows:

  1. Compute Euclidean or Mahalanobis distance from target plot to those that were sampled.
  2. Order samples taking for account calculated distances.
  3. Choose heuristically optimal k nearest neighbor based on RMSE done by cross validation technique.
  4. Calculate an inverse distance weighted average with the k-nearest multivariate neighbors.

Using a weighted k-NN also significantly improves the results: the class (or value, in regression problems) of each of the k nearest points is multiplied by a weight proportional to the inverse of the distance between that point and the point for which the class is to be predicted.

Read more about this topic:  k-nearest Neighbor Algorithm

Famous quotes containing the words estimating, continuous and/or variables:

    “I am sure that in estimating every man’s value either in private or public life, a pure integrity is the quality we take first into calculation, and that learning and talents are only the second.”
    —Thomas Jefferson (1743–1826)

    “The gap between ideals and actualities, between dreams and achievements, the gap that can spur strong men to increased exertions, but can break the spirit of others—this gap is the most conspicuous, continuous land mark in American history. It is conspicuous and continuous not because Americans achieve little, but because they dream grandly. The gap is a standing reproach to Americans; but it marks them off as a special and singularly admirable community among the world’s peoples.”
    —George F. Will (b. 1941)

    “Science is feasible when the variables are few and can be enumerated; when their combinations are distinct and clear. We are tending toward the condition of science and aspiring to do it. The artist works out his own formulas; the interest of science lies in the art of making science.”
    —Paul Valéry (1871–1945)