Trends in Random Data
Before considering trends in real data, it is useful to understand trends in random data.
If a series which is known to be random is analysed – fair dice falls, or computer-generated pseudo-random numbers – and a trend line is fitted through the data, the chances of an exactly zero estimated trend are negligible. But the trend would be expected to be small. If an individual series of observations is generated from simulations that employ a given variance of noise that equals the observed variance of our data series of interest, and a given length (say, 100 points), a large number of such simulated series (say, 100,000 series) can be generated. These 100,000 series can then be analysed individually to calculate estimated trends in each series, and these results establish a distribution of estimated trends that are to be expected from such random data – see diagram. Such a distribution will be normal according to the central limit theorem except in pathological cases. A level of statistical certainty, S, may now be selected – 95% confidence is typical; 99% would be stricter, 90% looser – and the following question can be asked: what is the borderline trend value V that would result in S% of trends being between −V and +V?
The above procedure can be replaced by a permutation test. For this, the set of 100,000 generated series would be replaced by 100,000 series constructed by randomly shuffling the observed data series; clearly such a constructed series would be trend-free, so as with the approach of using simulated data these series can be used to generate borderline trend values V and −V.
In the above discussion the distribution of trends was calculated by simulation, from a large number of trials. In simple cases (normally distributed random noise being a classic) the distribution of trends can be calculated exactly without simulation.
The range (−V, V) can be employed in deciding whether a trend estimated from the actual data is unlikely to have come from a data series that truly has a zero trend. If the estimated value of the regression parameter a lies outside this range, such a result could have occurred in the presence of a true zero trend only, for example, one time out of twenty if the confidence value S=95% was used; in this case, it can be said that, at degree of certainty S, we reject the null hypothesis that the true underlying trend is zero.
However, note that whatever value of S we choose, then a given fraction, 1 − S, of truly random series will be declared (falsely, by construction) to have a significant trend. Conversely, a certain fraction of series that in fact have a non-zero trend will not be declared to have a trend.
Read more about this topic: Trend Estimation
Famous quotes containing the words trends in, trends, random and/or data:
“Thanks to recent trends in the theory of knowledge, history is now better aware of its own worth and unassailability than it formerly was. It is precisely in its inexact character, in the fact that it can never be normative and does not have to be, that its security lies.”
—Johan Huizinga (1872–1945)
“A point has been reached where the peoples of the Americas must take cognizance of growing ill-will, of marked trends toward aggression, of increasing armaments, of shortening tempers—a situation which has in it many of the elements that lead to the tragedy of general war.... Peace is threatened by those who seek selfish power.”
—Franklin D. Roosevelt (1882–1945)
“Assemble, first, all casual bits and scraps
That may shake down into a world perhaps;
People this world, by chance created so,
With random persons whom you do not know—”
—Robert Graves (1895–1985)
“This city is neither a jungle nor the moon.... In long shot: a cosmic smudge, a conglomerate of bleeding energies. Close up, it is a fairly legible printed circuit, a transistorized labyrinth of beastly tracks, a data bank for asthmatic voice-prints.”
—Susan Sontag (b. 1933)