Anecdotal Evidence - Anecdotal Evidence and Faulty Logic

Anecdotal Evidence and Faulty Logic

Anecdotal evidence is often unscientific or pseudoscientific because various forms of cognitive bias may affect the collection or presentation of evidence. For instance, someone who claims to have had an encounter with a supernatural being or alien may present a very vivid story, but this is not falsifiable. This phenomenon can also happen to large groups of people through subjective validation.

Anecdotal evidence is also frequently misinterpreted via the availability heuristic, which leads to an overestimation of prevalence. Where a cause can be easily linked to an effect, people overestimate the likelihood of the cause having that effect (availability). In particular, vivid, emotionally-charged anecdotes seem more plausible, and are given greater weight. A related issue is that it is usually impossible to assess for every piece of anecdotal evidence, the rate of people not reporting that anecdotal evidence in the population.

A common way anecdotal evidence becomes unscientific is through fallacious reasoning such as the Post hoc ergo propter hoc fallacy, the human tendency to assume that if one event happens after another, then the first must be the cause of the second. Another fallacy involves inductive reasoning. For instance, if an anecdote illustrates a desired conclusion rather than a logical conclusion, it is considered a faulty or hasty generalization. For example, here is anecdotal evidence presented as proof of a desired conclusion:

There's abundant proof that drinking water cures cancer. Just last week I read about a girl who was dying of cancer. After drinking water she was cured.

Anecdotes like this do not prove anything. In any case where some factor affects the probability of an outcome, rather than uniquely determining it, selected individual cases prove nothing; e.g. "my grandfather smoked 40 a day until he died at 90" and "my sister never went near anyone who smoked but died of lung cancer". Anecdotes often refer to the exception, rather than the rule: "Anecdotes are useless precisely because they may point to idiosyncratic responses." Even when many anecdotes are collected to prove a point, "The plural of anecdote is not data." (Roger Brinner)

More generally, a statistical correlation between things does not in itself prove that one causes the other (a causal link). A study found that television viewing was strongly correlated with sugar consumption, but this does not prove that viewing causes sugar intake (or viceversa).

In medicine anecdotal evidence is also subject to placebo effects: it is well-established that a patient's (or doctor's) expectation can genuinely change the outcome of treatment. Only double-blind randomized placebo-controlled clinical trials can confirm a hypothesis about the effectiveness of a treatment independently of expectations.

Sites devoted to rhetoric often give explanations along these lines:

Anecdotal evidence, for example, is by definition less statistically reliable than other sorts of evidence, and explanations do not carry the weight of authority. But both anecdotal evidence and explanations may affect our understanding of a premise, and therefore influence our judgment. The relative strength of an explanation or an anecdote is usually a function of its clarity and applicability to the premise it is supporting. —

By contrast, in science and logic, the "relative strength of an explanation" is based upon its ability to be tested, proven to be due to the stated cause, and verified under neutral conditions in a manner that other researchers will agree has been performed competently, and can check for themselves.