Real Data Need More Complicated Models
Thus far the data have been assumed to consist of the trend plus noise, with the noise at each data point being independent and identically distributed random variables and to have a normal distribution. Real data (for example climate data) may not fulfill these criteria. This is important, as it makes an enormous difference to the ease with which the statistics can be analysed so as to extract maximum information from the data series. If there are other non-linear effects that have a correlation to the independent variable (such as cyclic influences), the use of least-squares estimation of the trend is not valid. Also where the variations are significantly larger than the resulting straight line trend, the choice of start and end points can significantly change the result. That is, the result is mathematically inconsistent. Statistical inferences (tests for the presence of trend, confidence intervals for the trend, etc.) are invalid unless departures from the standard assumptions are properly accounted for, for example as follows:
- Dependence: autocorrelated time series might be modeled using autoregressive moving average models.
- Non-constant variance: in the simplest cases weighted least squares might be used.
- Non-normal distribution for errors: in the simplest cases a generalised linear model might be applicable.
- Unit root: taking first differences of the data
Read more about this topic: Trend Estimation
Famous quotes containing the words real, data, complicated and/or models:
“Mother goddesses are just as silly a notion as father gods. If a revival of the myths of these cults gives woman emotional satisfaction, it does so at the price of obscuring the real conditions of life. This is why they were invented in the first place.”
—Angela Carter (1940–1992)
“To write it, it took three months; to conceive it three minutes; to collect the data in it—all my life.”
—F. Scott Fitzgerald (1896–1940)
“The ambiguous, gray areas of authority and responsibility between parents and teachers exacerbate the distrust between them. The distrust is further complicated by the fact that it is rarely articulated, but usually remains smoldering and silent.”
—Sara Lawrence Lightfoot (20th century)
“French rhetorical models are too narrow for the English tradition. Most pernicious of French imports is the notion that there is no person behind a text. Is there anything more affected, aggressive, and relentlessly concrete than a Parisan intellectual behind his/her turgid text? The Parisian is a provincial when he pretends to speak for the universe.”
—Camille Paglia (b. 1947)