Falsifiable

Many Medieval and Renaissance paintings show Adam and Eve with a leafy vine across their stomachs. The vine avoided theological controversy (and thus the Inquisition): Adam and Eve, having been created rather than born, required no navel, yet in creating prototypes would not God make his first humans like all the rest to follow? Subsequent humans would look the same whichever answer is correct.

A scientific theory must be consistent with facts, but neither facts nor experiments can prove a theory. First, conclusive proof would require conducting every possible experiment at every place and time. This is impossible. Second, a fact, or even any set of facts, may be consistent with more than one theory. The possibility of a new and better theory always exists. On the other hand, if a repeated experiment or event contradicts a theory, this is strong evidence refuting it. For these reasons, Karl Popper argued that to be scientific a claim must be disprovable by experiment or observation. The theory that best resists falsification stands—for the moment. The frequently heard but absurd claim that the “exception proves the rule” comes from a mistranslation of probar, and should read “the exception tests the rule.”

Consider the dispute between Antoine Lavoisier and Joseph Priestly. In the 1770s, both conducted the same experiment, heating ore of mercury. Lavoisier interpreted what he saw as an unknown substance given off by the ore. Priestly, an equally famous and capable scientist, interpreted what he saw as the ore absorbing a substance from the air.

Which explanation is true? Or, are both true? Or, for that matter, are both false? That is, there are four ways to account for the observation. How do you decide what is right? Long before Popper, Priestly and Lavoisier took the same course, trying instead of proving himself right to prove the other wrong, ultimately leaving Lavoisier’s oxygen a better explanation of the observation than Priestly’s phlogiston.

The effort to falsify characterizes science and underlies the logic of statistical research designed to determine whether an independent variable, or cause, affects a dependent variable, or result. Without getting into the particulars, inferential statistics determine whether to “retain” or “reject” a “null hypothesis” that there is no difference between groups subjected to different treatments. Retaining the hypothesis suggests that the independent variable does not have the effect being tested. Rejecting the null hypothesis increases the odds that it does.

Deciding something is false when it is true is termed a Type I error (Figure 1.1). A simple example is a union reducing its aspirations on the incorrect assumption that a high wage demand will lead to an unwanted strike. Deciding something is true when it is false is a Type II error. A simple example is making a wage demand lower than management would accept to avoid a strike.

Other things remaining equal, it turns out that the only way of reducing the chance of making one type of error increases the chance of making the other. Therefore, one has to decide in advance and reduce the chance of making whichever error is worse. In the case of labor negotiations, there are so many ways to back off from an excessive demand and so many long-term repercussions for not testing just how far management will go that unions usually want to avoid Type I errors more than they want to avoid Type II errors. This is true of many other bargaining situations as well, one explanation why initial positions tend to be extreme.

Knowing that science works by trying to prove theories false, careful researchers seek to rebut objections to their own theories before publication. The practice is not limited to the natural sciences. Donald Foster tried to identify the parts Shakespeare acted himself by linguistic analysis. He found that rarely used words first appeared in the dialogue of a single character.

Foster then tried to eliminate objections to his theory. He proved that the test never assigned Shakespeare to an improbable role, such as a child. He proved that unusual words were never scattered in an early play but clustered in a later one. Where the test assigns two roles to Shakespeare in the same play, he verified that they were never on stage at the same time. The theory survived these tests, leaving it the best current solution to this small but intriguing problem. More significantly, Foster used his technique to identify the author of Primary Colors, to help police identity the Unabomber (Roberts 2002), and to identify Eric Rudolph as the culprit in the 1996 Olympic bombing incident.

Sometimes errors are so sweeping that no rational person can continue to accept a theory. Examples of falsified theories are Afrocentrism (Howe 1998, Lefkowitz and Rogers 1996), the Aristotelian theory of gravity, astrology, the ether, fluidium, Lamarckism, the miasma theory of disease, the plum pudding model of the atom, and spontaneous generation.

Falsification is stronger than confirmation, but also is open to error. “Black Swans (Taleb 2007) such as Pearl Harbor and 9/11 are unpredictable events that have massive impact for which rational explanations are concocted after the fact.⁹ Some are unlikely to repeat but others require reconsidering our theories. Distinguishing the two possibilities is not always easy.