Real Data Need More Complicated Models
Thus far the data have been assumed to consist of the trend plus noise, with the noise at each data point being independent and identically distributed random variables and to have a normal distribution. Real data (for example climate data) may not fulfill these criteria. This is important, as it makes an enormous difference to the ease with which the statistics can be analysed so as to extract maximum information from the data series. If there are other non-linear effects that have a correlation to the independent variable (such as cyclic influences), the use of least-squares estimation of the trend is not valid. Also where the variations are significantly larger than the resulting straight line trend, the choice of start and end points can significantly change the result. That is, the result is mathematically inconsistent. Statistical inferences (tests for the presence of trend, confidence intervals for the trend, etc.) are invalid unless departures from the standard assumptions are properly accounted for, for example as follows:
- Dependence: autocorrelated time series might be modeled using autoregressive moving average models.
- Non-constant variance: in the simplest cases weighted least squares might be used.
- Non-normal distribution for errors: in the simplest cases a generalised linear model might be applicable.
- Unit root: taking first differences of the data
Read more about this topic: Trend Estimation
Famous quotes containing the words real, data, complicated and/or models:
“Sometimes among our more sophisticated, self-styled intellectuals—and I say self-styled advisedly; the real intellectual I am not sure would ever feel this way—some of them are more concerned with appearance than they are with achievement. They are more concerned with style then they are with mortar, brick and concrete. They are more concerned with trivia and the superficial than they are with the things that have really built America.”
—Lyndon Baines Johnson (1908–1973)
“Mental health data from the 1950’s on middle-aged women showed them to be a particularly distressed group, vulnerable to depression and feelings of uselessness. This isn’t surprising. If society tells you that your main role is to be attractive to men and you are getting crow’s feet, and to be a mother to children and yours are leaving home, no wonder you are distressed.”
—Grace Baruch (20th century)
“Progress celebrates Pyrrhic victories over nature. Progress makes purses out of human skin. When people were traveling in mail coaches, the world got ahead better than it does now that salesmen fly through the air. What good is speed if the brain has oozed out on the way? How will the heirs of this age be taught the most basic motions that are necessary to activate the most complicated machines? Nature can rely on progress; it will avenge it for the outrage it has perpetrated on it.”
—Karl Kraus (1874–1936)
“Friends broaden our horizons. They serve as new models with whom we can identify. They allow us to be ourselves—and accept us that way. They enhance our self-esteem because they think we’re okay, because we matter to them. And because they matter to us—for various reasons, at various levels of intensity—they enrich the quality of our emotional life.”
—Judith Viorst (20th century)