Contrasting Least Squares With Least Absolute Deviations
The following is a table contrasting some properties of the method of least absolute deviations with those of the method of least squares (for non-singular problems).
| Least Squares Regression | Least Absolute Deviations Regression | |
|---|---|---|
| Not very robust | Robust | |
| Stable solution | Unstable solution | |
| Always one solution | Possibly multiple solutions | |
The method of least absolute deviations finds applications in many areas, due to its robustness compared to the least squares method. Least absolute deviations is robust in that it is resistant to outliers in the data. This may be helpful in studies where outliers may be safely and effectively ignored. If it is important to pay attention to any and all outliers, the method of least squares is a better choice.
The instability property of the method of least absolute deviations means that, for a small horizontal adjustment of a datum, the regression line may jump a large amount. The method has continuous solutions for some data configurations; however, by moving a datum a small amount, one could "jump past" a configuration which has multiple solutions that span a region. After passing this region of solutions, the least absolute deviations line has a slope that may differ greatly from that of the previous line. In contrast, the least squares solutions is stable in that, for any small adjustment of a data point, the regression line will always move only slightly; that is, the regression parameters are continuous functions of the data.
Lastly, for a given data set, the method of least absolute deviations may produce multiple solutions, whereas the method of least squares always produces only one solution (the regression line is unique).
For a set of applets that demonstrate these differences, see the following site: http://www.math.wpi.edu/Course_Materials/SAS/lablets/7.3/73_choices.html
Read more about this topic: Least Absolute Deviations
Famous quotes containing the words contrasting, squares and/or absolute:
“We could not help contrasting the equanimity of Nature with the bustle and impatience of man. His words and actions presume always a crisis near at hand, but she is forever silent and unpretending.”
—Henry David Thoreau (1817–1862)
“And New York is the most beautiful city in the world? It is not far from it. No urban night is like the night there.... Squares after squares of flame, set up and cut into the aether. Here is our poetry, for we have pulled down the stars to our will.”
—Ezra Pound (1885–1972)
“In our science and philosophy, even, there is commonly no true and absolute account of things. The spirit of sect and bigotry has planted its hoof amid the stars. You have only to discuss the problem, whether the stars are inhabited or not, in order to discover it.”
—Henry David Thoreau (1817–1862)