Overview
Mathematician Keith Devlin provides an illustration of the risks of committing, and the challenges of avoiding, the base rate fallacy. He asks us to imagine that there is a type of cancer that afflicts 1% of all people. A doctor then says there is a test for that cancer which is about 80% reliable. He also says that the test provides a positive result for 100% of people who have the cancer, but it is also results in a 'false positive' for 20% of people - who actually do not have the cancer. Now, if we test positive, we may be tempted to think it is 80% likely that we have the cancer. Devlin explains that, in fact, our odds are less than 5%. What is missing from the jumble of statistics is the most relevant base rate information. We should ask the doctor "Out of the number of people who test positive at all (this is the base rate group that we care about), how many end up actually having the cancer?". Naturally, in assessing the probability that a given individual is a member of a particular class, we must account for other information besides the base rate. In particular, we must account for featural evidence. For example, when we see a person wearing a white doctor's coat and stethoscope, and prescribing medication, we have evidence which may allow us to conclude that the probability of this particular individual being a "medical professional" is considerably greater than the category base rate of 1%.
The normative method for integrating base rates (prior probabilities) and featural evidence (likelihoods) is given by Bayes rule. A large number of psychological studies have examined a phenomenon called base-rate neglect in which category base rates are not integrated with featural evidence in the normative manner.
Read more about this topic: Base Rate