Criticism
Stepwise regression procedures are used in data mining, but are controversial. Several points of criticism have been made.
- The tests themselves are biased, since they are based on the same data. (Rencher and Pun, 1980, Copas, 1983). Wilkinson and Dalall (1981) computed percentage points of the multiple correlation coefficient by simulation and showed that a final regression obtained by forward selection, said by the F-procedure to be significant at 0.1% was in fact only significant at 5%.
- When estimating the degrees of freedom, the number of the candidate independent variables from the best fit selected is smaller than the total number of final model variables, causing the fit to appear better than it is when adjusting the r2 value for the number of degrees of freedom. It is important to consider how many degrees of freedom have been used in the entire model, not just count the number of independent variables in the resulting fit.
- Models that are created may be too-small than the real models in the data.
Critics regard the procedure as a paradigmatic example of data dredging, intense computation often being an inadequate substitute for subject area expertise. Additionally, the results of stepwise regression are often used incorrectly without adjusting them for the fact that model selection has occurred. Especially the practice of fitting the final selected model as if no model selection had taken place and reporting of estimates and confidence intervals as if least-squares theory were valid for them, has been described as a scandal. Widespread incorrect usage and the availability of alternatives such as Ensemble learning, leaving all variables in the model or using expert judgement to identify relevant variables have lead to calls to totally avoid stepwise model selection.
Read more about this topic: Stepwise Regression
Famous quotes containing the word criticism:
“To be just, that is to say, to justify its existence, criticism should be partial, passionate and political, that is to say, written from an exclusive point of view, but a point of view that opens up the widest horizons.”
—Charles Baudelaire (1821–1867)
“The aim of all commentary on art now should be to make works of art—and, by analogy, our own experience—more, rather than less, real to us. The function of criticism should be to show how it is what it is, even that it is what it is, rather than to show what it means.”
—Susan Sontag (b. 1933)
“Unless criticism refuses to take itself quite so seriously or at least to permit its readers not to, it will inevitably continue to reflect the finicky canons of the genteel tradition and the depressing pieties of the Culture Religion of Modernism.”
—Leslie Fiedler (b. 1917)