Criticism
Stepwise regression procedures are used in data mining, but are controversial. Several points of criticism have been made.
- The tests themselves are biased, since they are based on the same data. (Rencher and Pun, 1980, Copas, 1983). Wilkinson and Dalall (1981) computed percentage points of the multiple correlation coefficient by simulation and showed that a final regression obtained by forward selection, said by the F-procedure to be significant at 0.1% was in fact only significant at 5%.
- When estimating the degrees of freedom, the number of the candidate independent variables from the best fit selected is smaller than the total number of final model variables, causing the fit to appear better than it is when adjusting the r2 value for the number of degrees of freedom. It is important to consider how many degrees of freedom have been used in the entire model, not just count the number of independent variables in the resulting fit.
- Models that are created may be too-small than the real models in the data.
Critics regard the procedure as a paradigmatic example of data dredging, intense computation often being an inadequate substitute for subject area expertise. Additionally, the results of stepwise regression are often used incorrectly without adjusting them for the fact that model selection has occurred. Especially the practice of fitting the final selected model as if no model selection had taken place and reporting of estimates and confidence intervals as if least-squares theory were valid for them, has been described as a scandal. Widespread incorrect usage and the availability of alternatives such as Ensemble learning, leaving all variables in the model or using expert judgement to identify relevant variables have lead to calls to totally avoid stepwise model selection.
Read more about this topic: Stepwise Regression
Famous quotes containing the word criticism:
“Cubism had been an analysis of the object and an attempt to put it before us in its totality; both as analysis and as synthesis, it was a criticism of appearance. Surrealism transmuted the object, and suddenly a canvas became an apparition: a new figuration, a real transfiguration.”
—Octavio Paz (b. 1914)
“The visual is sorely undervalued in modern scholarship. Art history has attained only a fraction of the conceptual sophistication of literary criticism.... Drunk with self-love, criticism has hugely overestimated the centrality of language to western culture. It has failed to see the electrifying sign language of images.”
—Camille Paglia (b. 1947)
“Parents sometimes feel that if they don’t criticize their child, their child will never learn. Criticism doesn’t make people want to change; it makes them defensive.”
—Laurence Steinberg (20th century)