Critics
In 2004, the question How Much Should We Trust Differences-in-Differences Estimates? was asked in an article with the same name, and apparently the answer is "not all that much." It is worth noting however that this is only the case if DiD is employed without considering the auto-correlation in calculating the standard errors. Most papers that employ Difference-in-Differences estimation use many years of data and focus on serially correlated outcomes but ignore that the resulting standard errors are inconsistent, leading to serious over-estimation of t-statistics and significance levels. These conventional DID standard errors severely understate the standard deviation of the estimators: we find an "effect" significant at the 5 percent level for up to 45 percent of the placebo interventions. To alleviate this problem two corrections based on asymptotic approximation of the variance-covariance matrix work well for moderate numbers of states and one correction that collapses the time series information into a "pre" and "post" period and explicitly takes into account the effective sample size works well even for small numbers of states.
Read more about this topic: Difference In Differences
Famous quotes containing the word critics:
“Neither can I do anything to please critics belonging to the good old school of “projected biography,” who examine an author’s work, which they do not understand, through the prism of his life, which they do not know.”
—Vladimir Nabokov (1899–1977)
“Individual science fiction stories may seem as trivial as ever to the blinder critics and philosophers of today—but the core of science fiction, its essence ... has become crucial to our salvation if we are to be saved at all.”
—Isaac Asimov (1920–1992)
“Some critics are like chimneysweepers; they put out the fire below, and frighten the swallows from the nests above; they scrape a long time in the chimney, cover themselves with soot, and bring nothing away but a bag of cinders, and then sing out from the top of the house, as if they had built it.”
—Henry Wadsworth Longfellow (1807–1882)