Sampling Frame - Sampling Frames Problems

Sampling Frames Problems

The sampling frame must be representative of the population and this is a question outside the scope of statistical theory demanding the judgement of experts in the particular subject matter being studied. All the above frames omit some people who will vote at the next election and contain some people who will not; some frames will contain multiple records for the same person. People not in the frame have no prospect of being sampled.

Because a cluster-based frame contains less information about the population, it may place constraints on the sample design, possibly requiring the use of less efficient sampling methods and/or making it harder to interpret the resulting data.

Statistical theory tells us about the uncertainties in extrapolating from a sample to the frame. It should be expected that sample frames, will always contain some mistakes. In some cases, this may lead to sampling bias. Such bias should be minimized, and identified, although avoiding it completely in a real world is nearly impossible. One should also not assume that sources which claim to be unbiased and representative are such.

In defining the frame, practical, economic, ethical, and technical issues need to be addressed. The need to obtain timely results may prevent extending the frame far into the future. The difficulties can be extreme when the population and frame are disjoint. This is a particular problem in forecasting where inferences about the future are made from historical data. In fact, in 1703, when Jacob Bernoulli proposed to Gottfried Leibniz the possibility of using historical mortality data to predict the probability of early death of a living man, Gottfried Leibniz recognized the problem in replying:

Nature has established patterns originating in the return of events but only for the most part. New illnesses flood the human race, so that no matter how many experiments you have done on corpses, you have not thereby imposed a limit on the nature of events so that in the future they could not vary. —Gottfried Leibniz

Kish posited four basic problems of sampling frames:

1. Missing elements: Some members of the population are not included in the frame.
2. Foreign elements: The non-members of the population are included in the frame.
3. Duplicate entries: A member of the population is surveyed more than once.
4. Groups or clusters: The frame lists clusters instead of individuals.

Problems like those listed can be identified by the use of pre-survey tests and pilot studies.