1983 ISIS Survey - Response

Response

For the most part, mainstream scientists have paid very little attention to Cameron's studies, and thus extensive scientific analysis of the ISIS Survey have not been widely available. Dr. Gregory Herek, Ph.D., is among those who have analyzed Cameron's findings. Among the errors Dr. Herek encountered were:

• Mischaracterized sample: "Although the Cameron group has claimed that theirs was a "national" sample and have repeatedly used their data to make generalizations about the entire population, the initial sampling frame consisted only of 7 municipalities"
• Unacceptably low response rate: "the Cameron group's results cannot be considered representative of even the specific municipalities because the vast majority of their sample did not complete the survey."
• Unreliable analyses due to small subsamples: "If the Cameron group's combined 1983-84 sample had been a random national sample (which it was not, as explained above), its size (N = 5,182 people) would have been large enough to permit estimates of population characteristics with only a small margin of error. Because their extremely low response rate rules out the possibility of making any population estimates on the basis of their data, however, this point is moot."
• Questionable validity: "It included a large number of questions that dealt with highly sensitive aspects of sexuality, many of them presented in an extremely complicated format. This procedure raises concerns about respondent fatigue and item difficulty."
• Biased interview procedures: "The Cameron group's reports gave no information about how interviewers were trained or supervised in the field."
• Researcher's bias publicized during data collection: "One of the principal challenges of social research is that the individuals who are being studied can become aware of the researcher's expectations or goals, which can alter their behavior. For this reason, researchers do not communicate their expectations or hypotheses in advance to research participants. Nor do they bias participants' responses by suggesting that a particular answer is more correct or desirable than others. Contrary to this well-established norm, Paul Cameron publicly disclosed the survey's goals and his own political agenda in the local newspaper of at least one surveyed city (Omaha) while data collection was in process."

Herek's conclusion was that "an empirical study manifesting even one of these six weaknesses would be considered seriously flawed. In combination, the multiple methodological problems evident in the Cameron group's surveys mean that their results cannot even be considered a valid description of the specific group of individuals who returned the survey questionnaire."