Cosmic Variance - Philosophical Issues

Philosophical Issues

This raises philosophical problems: suppose that random physical processes happen on length scales both smaller than and bigger than the horizon. A physical process (such as an amplitude of a primordial perturbation in density) that happens on the horizon scale only gives us one observable realization. A physical process on a larger scale gives us zero observable realizations. A physical process on a slightly smaller scale gives us a small number of realizations.

In the case of only one realization it is difficult to draw statistical conclusions about its significance. For example, if the underlying model of a physical process implies that the observed property should occur only 1% of the time, does that really mean that the model is excluded? Consider the physical model of the citizenship of human beings in the early 21st century, where about 30% are Indian and Chinese citizens, about 5% are American citizens, about 1% are French citizens, and so on. For an observer who has only one observation (of his/her own citizenship) and who happens to be French and cannot make any external observations, the model can be rejected at the 99% significance level. Yet the external observers with more information unavailable to the first observer, know that the model is correct.

In other words, even if the bit of the universe observed is the result of a statistical process, the observer can only view one realization of that process, so our observation is statistically insignificant for saying much about the model, unless the observer is careful to include the variance. This variance is called the cosmic variance and is separate from other sources of experimental error: a very accurate measurement of only one value drawn from a distribution still leaves considerable uncertainty about the underlying model. Variance is normally plotted separately from other sources of uncertainty. Because it is necessarily a large fraction of the signal, workers must be very careful in interpreting the statistical significance of measurements on scales close to the horizon.

In physical cosmology, the common way of dealing with this on the horizon scale and on slightly sub-horizon scales (where the number of occurrences is greater than one but still quite small), is to explicitly include the variance of very small statistical samples (Poisson distribution) when calculating uncertainties. This is important in describing the low multipoles of the cosmic microwave background and has been the source of much controversy in the cosmology community since the COBE and WMAP measurements.