Base Rate

Tuomas W. Manninen

According to the National Crime Victimization Survey by the Bureau of Justice Statistics, between 1999 and 2011, the number of whites who were killed by the police was 2151, which is nearly twice the number of blacks killed by the police during the same period, 1130.

Collected from the Internet

The base rate fallacy is a fallacy that occurs in probabilistic reasoning when available general information (the base rate, which pertains to a population as a whole) is omitted from the calculations - either accidentally or deliberately - and attention is given to specific information only (e.g., information pertaining to the sample group) (FallacyFiles 2015).

Consider the following scenario in Philip K. Dick’s (1987) short story “Minority Report,” with some details augmented by the 2002 movie adapta­tion by Steven Spielberg. The felony crime rates in New York City in the 2050s have dropped by 99.8%, thanks to the “PreCrime” unit: the information made available to this unit by Precogs - individuals who can foresee criminal acts before they happen - allows the potential perpetrators to be apprehended before they commit the criminal act.

Setting aside many other questions about this approach - and the fact that neither Dick nor Spielberg fleshed out all the details - let us focus on the “99.8%” figure both for sensitivity and selectivity. That is, suppose that of the individuals detected as criminals by the PreCrime unit, a full 99.8% of them were going to commit a criminal act (selectivity, or the rate for true positives). Similarly, of the individuals identified as not-criminals, a full 99.8% of them are not-criminals (sensitivity, or the rate for true negatives). Having granted this, we have the 0.2% possibility for false positives (i.e., someone identified as a criminal when he is not) and for false negatives (i.e., someone identified as not-criminal when she actually is one) alike.

Looking at the numbers given above (99.8% versus 0.2%), it seems far more likely than not that you are guilty of a precrime if you are identified by the system. However, this appearance is misleading, because we do not know the base rate of criminal activity. Drawing from information in 2014, the New York City’s crime rate was 639.3 violent crimes per a population of 100,000, or 0.0065, and the population was 23,462,000 (“Crime in New York City”). Assuming these numbers remain the same for the scenario, we can apply Bayes’ Theorem to solve the question: What is the probability that an individual is guilty if she has been identified by PreCrime as a criminal? More formally, what is the value of p(guilty∕PreCrime+)?

Using the aforementioned percentage for the PreCrime’s sensitivity and selectivity, together with the base rate information (of how many crimes there are), we can represent the results in the following chart:

We could apply Bayes’ Theorem to make these determinations (i.e., P (A / B) = P (A) P (B / A) / P (B) for the conditional probability of event A, given that B).

Despite using the fictional example here, this problem comes up in various real-life situations. After all, no breathalyzer, drug test, personality test, crimi­nal profile, and so on, can claim to score 100% both in terms of selectivity

and in terms of sensitivity. As such, the seemingly high rate of accuracy of - or the seemingly devastating positive result from - these tests is not so much so, after all.

In recent years, many US state legislatures (Arizona, Kansas, Mississippi, Missouri, Oklahoma, Tennessee, and Utah) have enacted laws that require those who apply for Temporary Assistance for Needful Families (TANF) or for other welfare programs to pass a drug test as a condition of their eligibil­ity. The rationalization for such laws is that those who use illicit narcotics should be ineligible to receive government welfare; if an individual has enough income to dispose on narcotics, then the state should not provide him with funds to enable his illicit activities. Other states are also consider­ing similar legislation (e.g., Florida, Michigan, Wisconsin) despite the fact that the initial results have not been exactly encouraging; according to Covert and Israel’s (2015) analysis:

The statistics show that applicants actually test positive at a lower rate than the drug use of the general population. The national drug use rate is 9.4 percent. In these [seven aforementioned] states, however, the rate of positive drug tests to total welfare applicants ranges from 0.002 percent to 8.3 percent, but all except one have a rate below 1 percent.

The legislative approach seems to be based not just on the base rate fallacy but on assuming a base rate that is divorced from reality, that the drug use rate among welfare applicants is significantly higher than the national rate.

How do we avoid the fallacy? Let us return to the example from the epi­graph. Stated more formally, we can put the argument as follows:

(1) Between 1999 and 2011, the number of whites killed by the police was 2151.

(2) Between 1999 and 2011, the number of blacks killed by the police was 1130.

(3) Implied: 2151 is greater than 1130.

(4) Therefore, it seems inaccurate to speak of “‘white privilege” when it comes to police violence, because more whites than blacks were killed by the police.

Although the argument seems to be valid, it can be shown to be unsound for the reason that it deals with absolute numbers rather than numbers propor­tioned to the racial breakdown of the US population. If the US population was evenly divided by whites and blacks, then the argument would be sound. However, using the available figures from the US Census Bureau, we can see the argument’s unsoundness quite clearly. According to the 2010 Census numbers, the US population was 72.4% white and 12.6% black.

If we apply these percentages to the above argument, we get the following:

(1) Between 1999 and 2011, the number of whites killed by the police, in proportion to the population, was (2151/0.724) = 2971.

(2) Between 1999 and 2011, the number of blacks killed by the police, in proportion to the population, was (1130/0.126) = 8968.

(3) Implied: 8968 is greater than 2971.

(4) Therefore, it seems accurate to speak of “white privilege” when it comes to police violence, because more blacks than whites were killed by the police in proportion to the respective populations.

Thus, once the accurate base rate is applied (be it the proportions between the races in the US population, or the percentage of drug users among the people receiving government welfare, etc.), we can clearly see that the origi­nal conclusion does not follow.

References

Covert, Bryce, and Josh Israel. “What Seven States Discovered after Spending More than $1 Million Drug Testing Welfare Recipients.” ThinkProgress, February 26. http://thinkprogress.org/economy/2015/02/26/3624447/tanf-drug-testing-states/ (accessed September 26, 2017).

“Crime in New York City.” Wikipedia. https://en.wikipedia.org/wiki/Crime_in_ New_York_City (accessed October 22, 2017).

Dick, Philip. 1987. The Minority Report and Other Classic Stories. New York, NY: Kensington Publishing.

FallacyFiles. 2015. “The Base Rate Fallacy.” http://www.fallacyfiles.org/baserate. html (accessed September 26, 2017).