D What We Do and What We Should Do: The Hypothesis and Rationality

I took this detour into the nature of rationality because we're going to evaluate scientific bias in the context of rationality, and it is important to know about the two standards.

Earlier in this chapter, I noted that scientific thinking has been held up as a model for rational thinking in general, and yet everyone agrees that “bias,” as a form of mistaken thinking, creates problems for science. We've considered two viewpoints on how rationality and biases are related. One take on the conflict be­tween Gigerenzer's and Kahneman's theories is that they do not genuinely differ on substantial matters,³² “core claims” but are at odds in the arena of “rhetor­ical flourishes,” those flamboyant beliefs that each side holds but which it cannot fully support. Indeed, it often appears that they are arguing past each other about two separate questions: How do humans think, and how should they think? This is a variant of the split between “descriptive” (what actually happens) and “nor­mative” (what ought to happen) analyses that cognitive scientists, philosophers of science, and others often make. It implies that the teachings of both schools of thought are complementary and equally applicable, and, in one sense, this impli­cation seems entirely reasonable.

In another sense, the teachings are incompatible and, since we're interested in rationality and science, we need to know whether the descriptive or the nor­mative side is more important. We have to choose: Which is the most appro­priate for scientific thinking? The answer is: it depends on what you're trying to accomplish.

12.D.1 Critical Scientific Thinking Requires By-the-Book Logic

When it comes to critical scientific thinking, there can't be any real debate: we must be narrowly focused, logical, and rational in the traditional sense. As the hypothesis (and its cousins: theories and laws) is the premier logical structure in science, we are obliged to apply the psychological principles that take formal logic as the measure of rationality when we're evaluating hypothesis-based sci­entific thinking. If a hypothesis makes a prediction that is logically inconsistent with evidence, the hypothesis is false; if a prediction does not logically follow from a hypothesis, testing the prediction does not truly test the hypothesis, etc. There are right and wrong answers. Kahneman's standards for rational and irra­tional thinking are also ours when we design and test hypotheses. Conventional logic is what holds science together.

What constitutes a logical argument is not always unambiguous and scientists can disagree about whether a given test does or does not adequately test a hypo­thesis. Nevertheless, scientists believe that True answers do exist and that, guided by empirical evidence, we're capable of finding them (despite remaining unsure if we've been successful). The logical ideal is easy to state and hard to achieve, and one reason is that our automatic thinking generally doesn't conform to the demands of textbook logic. It adheres to the built-in guidelines that we've ac­quired through the ages.

12.D.2 Biases and Ecological Rationality

As many commentators have noted, scientific thinking is a learned skill, and the primary hurdle that we have to clear in acquiring it is unlearning, or learning to modify, our normal thinking habits. The first step must be to recognize that the problem exists.

The concept of ecological rationality goes a long way toward explaining why scientific thinking is so hard for so many of us. Ecological rationality gives rise to thinking that it is pragmatic and practically useful though not necessarily com­patible with scientific reasoning.

We should depend on heuristics only when they are appropriate and avoid them when they're not. When operating in their critical analytic mode, scientists gravitate toward the logical thinking standard. For instance, in the course of their staunch defense of ecological rationality, Gigerenzer and his colleagues offer co­gent critiques of, say, Prospect Theory, that conform to conventional logical rea­soning. They formulate, test, and evaluate their hypotheses in good agreement with the guidelines suggested in Chapter 2.

The first step in dealing with the potential problems caused by automatic, heuristic-based thinking must be to recognize when we need to switch into the rigorous logical mode. It would be helpful if we had an external signal to tell us when the scientific mode is called for. Putting on a white lab coat may help those of us who wear them to shift cognitive gears and get into the science-thinking frame of mind. Along this line, I've wondered (not too seriously) if maybe those of us who don't wear lab coats could put on a special “science-beanie” to prompt us to take a formal cognitive stance and abide by the rules of cold, abstract logic. You'd take off the beanie when you were being imaginative and dreaming up hypotheses or asking questions. Taking off the beanie would also signal to critics gnawing at the old worn tropes about how rigorous scientific thinking “stifles creativity” that all is well; disciplined as well as fancy-free thinking are both com­patible with doing good science, each in its time.

To streamline the following discussion, I will adopt Kahneman's terminology and position: deviations from logic are irrational or at least unfortunate when they happen.

12. D.2.a The Currency of Abstract Thought: What Have We Got to Lose? We all keep mental accounts of gains and losses³³ in many facets of our lives. “I owe you one,” we say when someone does us a favor; we incur a “debt of gratitude” and plan to “pay it back” to square the social account. Likewise, guilt, regret, pride, and self-respect are currencies in the self-image accounts that we try to keep in balance. Scientists, too, keep mental accounts, and their account balances can affect how they behave, whether it is their will­ingness to state their hypotheses or their tendency to act in biased ways. Our mental accounts are so important that innate cognitive alarms go off to warn of imbalances.

We do not like to suffer loss of any psychological goods; indeed, the mere ex­pectation of loss can trigger loss aversion; we suffer if we are to lose the possi­bility of a gain that we hadn't received, and a region of our brains is especially attuned to detecting loss.³⁴ Loss aversion strongly influences how we make risky decisions when we are uncertain, and uncertainty is a dominant motif in science. It is not that people are congenitally allergic to risk, it's just that we're inconsistent in how we confront it. The threat of loss on their mental accounts of self-esteem may affect scientists' behavior.

12.D.2.b Loss Aversion and the Hypothesis

I noted earlier (Chapter 9) that the majority of scientific papers that I reviewed steered away from an explicitly stated hypothesis, and my survey respondents cited a few of their reasons for not stating them. If you think about what we've been saying about self-images and mental account balancing, you might rec­ognize a reason. Let's suppose that scientists assess their self-esteem in units, call them creds, and that the better their reputation among their colleagues, the greater their self-esteem, the more creds in their mental accounts. Achieving success, publishing a good paper, getting a grant, being thought reliable and careful by their peers, all add to their accumulation of creds, while failure, being thought sloppy, unreliable, or untrustworthy, etc., deplete it.

Imagine that you have made a new discovery and have a novel hypothesis to explain it. As you write up your work for publication, you face a choice: Do you make your hypothesis explicit, or do you put forward a less-structured essay and hope that your readers will get the message anyway? If you don't state your hy­pothesis, then you might not get credit, or be cited, for it. (Citations, also known as “hits,” are published references to your work that acknowledge your contri­bution; citations are coins of the realm in science: they add creds to your ac­count.) If your hypothesis becomes widely accepted, you'll be better known and be invited to visit other places and give seminars about it, which are other ways of amassing creds. Contrariwise, not stating your hypothesis risks not getting all the creds that you deserve, and, what would be worse, if you don't state it explic­itly, someone else might and claim the glory (yes, this happens in real life). Such factors strongly favor not beating around the bush and getting your hypothesis out into the public eye.

On the other hand, the very thought of stating your hypothesis directly causes you anxiety; it could be a dicey move. Your competitors could take ad­vantage of the research plan so clearly laid out by your hypothesis and jump ahead of you. After all, its key predictions are dead obvious; you're virtually giving your opponents directions to follow and, maybe, prove you wrong. By putting your hypothesis out there in plain terms for all to see, you will have stuck your neck out; your fear of the loss of creds if it is falsified is almost pal­pable. Or, what if people think that you're being pretentious and making a big deal of a modest insight (“I hypothesize.. Does anybody really to do that, or will it somehow mark you as a phony? You begin to fret that the costs of stating your hypothesis may outweigh the benefits.

In this made up but not entirely far-fetched scenario, loss aversion, the fear of losing creds, determines how scientists present their results. Although scientists undoubtedly don't consciously go through such deliberations, unconscious ap­prehension about losing creds may well affect what they do. I'll come back to these concerns and how you can address them in Chapter 14.

12.D.3 Confirmation Bias: Loss Aversion, Cognitive Dissonance, or Ignorance?

Confirmation bias is the tendency to seek and hold onto information that agrees with, or “confirms” your hypotheses and to ignore disconfirming information. Confirmation bias is always bad and is often linked to having a hypothesis. Bizarrely enough, though, some critics imply that the problem lies with the hy­pothesis, not the bias itself. They say that if we want to reduce bias, we should stop doing hypothesis-based science. On the contrary, if you want to avoid con­firmation bias, failing to propose and test hypotheses is emphatically the wrong way to go.

12.D.3.a Confirmation Bias, Loss Aversion, and Cognitive Dissonance Loss aversion applied to personal creds can explain confirmation bias. We see ourselves as intelligent, thoughtful observers of nature and we expect our beliefs about it to be true. Evidence that flies in the face of our cherished hypotheses threatens a loss of creds, which we try to prevent by amassing confirmatory evi­dence and evading the disconfirmatory kind.

A psychological stimulus that can bolster a tendency toward confirmation bias is a need to reduce cognitive dissonance. Cognitive dissonance is the unpleasant feeling set up by “the state of tension that occurs whenever a person holds two cognitions (idea, attitudes, beliefs, opinions) that are psychologically incompat­ible.”³⁵ It is an emotion that arises from a threat to our sense of self-consistency and that impels us to reduce the threat. The archetypal case is a cigarette smoker who, knowing that smoking is bad for him, does one of two things: quits smoking or convinces himself that smoking is not so bad, that “everybody’s got to go sometime,” etc., and keeps on smoking. Both strategies bring him some peace of mind by reducing the dissonance. Cognitive dissonance is not restricted to justifying (“rationalizing”) unhealthy personal habits, however. Carol Tavris and Elliott Aronson³⁶ argue that the urge to reduce cognitive dissonance drives professional psychotherapists to concoct unfalsifiable hypotheses to defend their tenuous diagnoses. Would it be surprising if, for the same reason, a scien­tist whose ego got caught up in his hypothesis had trouble conceding that it was wrong and therefore searched for more reasons to think it was right?

A desire to circumvent the emotional tolls of loss aversion or cognitive disso­nance are possible contributors to confirmation bias, but ignorance is another one, although it is frequently overlooked.

12. D.3.b Confirmation Bias and the Hypothesis

Blaming the hypothesis for confirmation bias is exquisitely ironic because the originator of tests widely used to study confirmation bias, Peter Wason, attrib­uted the problem to a failure of his subjects to seek out potentially disconfirming evidence to test their hypotheses.³⁷ Here’s an example of one of his tests: you are given a set of three numbers—2, 4, and 8—that conforms to a rule, and you have to figure out the rule by gathering information and creating and testing hypoth­eses about what the rule is. To gather information, you propose a set of three numbers that you think might follow the rule and you’re told whether you’re right or not. You can continue like this, proposing sets of three numbers and getting feedback, for as long as you like; there is no time limit and no penalty for wrong proposals. After you’ve gathered enough information and believe that you know what the rule is, you get one chance to state your hypothesis about it. This is the payoff, the whole point of the exercise, so you want to be as sure as possible before committing yourself. In Wason’s day (1960s), his subjects used pencil and paper, and he provided the feedback on their preliminary proposals in person. You might be interested in trying an online version of the test before going fur­ther; it is easy to do, gives you real insight into the problem of confirmation bias, and is anonymous. Here's the link: https://www.nytimes.com/interactive/ 2015/07/03/upshot/a-quick-puzzle-to-test-your-problem-solving.html (Spoiler alert: I'm going to analyze the test next, so if you want to get the full effect, you should pause and try it now.)

Wason wanted to know if ordinary people (well, college psychology students, anyway) behaved as scientific reasoners were supposed to. Did they rigorously test their hypotheses before putting them forward? Wason designed his test in such a way that using inductive reasoning to generalize a rule from the sample sequence alone was virtually guaranteed to fail. Nonetheless, he found that most subjects were satisfied if they guessed a couple of correct sequences before they were confident enough to state their hypothesis, which was usually incorrect. The few subjects whose first hypothesis was correct guessed at many more sequences and, in particular, tried many nonconforming sequences before declaring their hypothesis about the true rule, which was simply “a series of three increasing numbers.”

Likewise, 77% of the participants in the online version failed to do enough exploration to find at least one wrong sequence of numbers before stating their hypothesis. In both classroom and online, the great majority of people quickly homed in on one pattern—for example “an increasing series of even numbers”— entered a few numerical sequences that conformed to their notion, and then declared their incorrect hypothesis.

A popular psychological interpretation of the Wason test results is that people want to hear that they are right and don't like to be told that they're wrong. Using inductive reasoning, they stick with what works. They don't suggest sets of num­bers that might be wrong to avoid what Wason calls “the disenchantment of negative instances.” The phenomena that we considered in the previous section, loss aversion and cognitive dissonance, probably factor into the “disenchant­ment,” although Wason does not pursue the matter. In any case, it is possible that subjects in his tests make only suggestions that they expect will earn them “yes” answers for emotional reasons.

No doubt there is some validity to this interpretation; however, Wason thought there must be more to it. Most people trying to solve a puzzle want more than a pat on the head and praise for making a good try: they really want to solve the puzzle. The goal in the Wason test is to find the true rule governing the series, and if you don't rigorously test your hypothesis you'll probably fail to find the rule. Surely, this intellectual failure would sting their egos more than proposing a nonconforming set of numbers would, so why didn't the students work harder to test their hypotheses?

To discover why, Wason asked them to talk through their reasoning out loud, and, furthermore, he had them try again if their first hypothesis was incorrect. Surprisingly, on hearing that they had failed, many subjects stuck with their orig­inal strategy or made only trivial changes before declaring another hypothesis. In the extreme cases, they simply restated their initial incorrect rule in slightly different words and without testing any more sets of numbers. Wason called this “magical thinking,” as in sorcery, where the success or failure of spells hinges on precisely how the words are spoken.

Wason concluded that the majority of the students were simply unaware of the power of falsification to test their hypotheses. '1 hey clung to their superficial inductive solutions, not because they were seeking emotional reassurance, but because they didn't know what else to do. Confirmation bias was the product of ignorance.

One bright note: Wason found that many students, on hearing that their first hypothesis about the proposed rule was wrong, did change their plan of attack and carefully check their next hypothesis with several negative and positive instances before submitting it. (Kahneman might say that failure jolted their System 2s into action.) However that may be, it is clear that critical scientific thinking habits can be strengthened by instruction and feedback. If this inter­pretation of the Wason test results is correct, then the solution to the problem of confirmation bias is not less hypothesizing, but an increased emphasis on testing hypotheses by trying to falsify them.

Kahneman thinks about confirmation bias itself in different terms.³⁸ He says that the bias arises because our System 1, in trying to make sense of the world, first looks for reasons to believe that a proposition is true. 'lhe concept is that if you can't believe a statement is true, you can't very well judge if it's true. Only after System 1 has understood a statement by believing it can our analytical System 2 scrutinize it and decide whether it actually is true or not. Unfortunately, System 2 is lazy, and skeptical examination is hard, so without determined effort to engage System 2, we are often stuck with what System 1 comes up with—an apparently “confirmed” proposition that we haven't thoroughly vetted. Improving our scien­tific thinking skill requires rousing our System 2 and forcibly directing its atten­tion toward the task at hand. (Maybe the beanie would help?)

As I've mentioned, despite rejecting traditional standards of rationality, the fast-and-frugal school of thought does recognize that people at times must act in accordance with conventional logic. We face two major stumbling blocks when we try to solve problems in logic and probability. first, these kinds of problems are typically cast in abstract, unnatural language that we can't process, and second, our educational systems don't adequately prepare us to think log­ically. Along these lines, Gigerenzer has championed a push to make biomed­ical communication more understandable to patients and doctors by switching to frequency-based language³⁹—think about the red-nosed liars. And he argues that, starting in the early school years, teaching the fundamentals of probability thinking in the tangible terms of natural frequencies can improve society's risk literacy.⁴⁰

I noted that others⁴¹ have called attention to the areas of convergence of the programs of Kahneman and Gigerenzer on some of the practical issues sur­rounding rationality. I'd like to draw the optimistic conclusion that we can im­prove our ability to think scientifically, provided that we start by acknowledging that, for whatever reason, we're not born scientific thinkers and be willing to exert concerted effort.

12.D.4 “It Ain't What You Say, It's the Way That You Say It... Framing Effects Matter

Those of us of a certain age remember the early days of credit cards when you had to pay an additional amount, a “surcharge,” if you wanted to use a card to buy an item in a store. Since then, things have changed, yet, in a way, not much is different. Nowadays, if you fill up your car's tank at your local gas station, you are likely to get a “discount” if you pay with cash. There is never a word about surcharges, though the end result is the same as it was in the early days—you pay more if you pay by card. If the net effect is the same, why did the language change?

Richard Thaler,⁴² the Nobel Prize-winning economist, associated this overwhelming—though strictly speaking irrational—preference for the revised wording with a phenomenon he named the “endowment effect”; we really hate to give up what we already have. He explains that it is relatively easy to tolerate not getting a discount and paying the “regular price” by credit because, well, we don't exactly own the discount, but we definitely do own our money and do not want to have to “pay extra.” Kahneman and Tversky call it framing when we let the in­dividual words of a choice, rather than its overall meaning, determine our prefer­ence. Framing effects seem to influence our behavior in countless ways.

12.D.5 Framing Effects, Loss Aversion, and Publication Bias

“Negative data” are the Rodney Dangerfield of scientific results: they get no respect.⁴³ No one wants to publish them, which is a major reason for publica­tion bias. The journals are full of positive research findings, while negative ones languish, forgotten in file drawers. The bias skews the literature and prevents scientists from learning about previously discovered dead ends, thus slowing progress and wasting resources, as others rediscover what doesn't work. Setting policies, haranguing people, or offering incentives and rewards for publishing negative data will probably help reduce publication bias, but we could do more if we took into account why scientists make their decisions. Besides disincentives, what factors might make a scientist hesitate to publish negative results, especially ones that are maligned as “failures” because they falsify hypotheses? Framing effects and loss aversion may be involved

The power of negative connotations of negative data and failure shouldn't be underestimated. After all, if positive data and successes are the shining ideals of science, what good are negative data and failures? At best they are unworthy goals; at worst, they're entirely pointless. Why would anyone spend time and ef­fort pursuing them? Who wants to be the guy who seeks them out; how many creds do you lose if you publish such stuff? I can't prove that scientists enter­tain these very thoughts but, given the effort that they spend building up their stores of creds, it is likely that framing effects play into the bias against publishing negative data.

If such considerations do influence our mental calculus, it should be relatively cost-free, although not necessarily easy, to counteract them: we can stop de­grading the value of negative data by not associating them with failure and loss, and we can start touting their value in identifying fruitless avenues of research and eliminating inadequate explanations. We can counter the concern that neg­ative data will clog the journals by distinguishing between mindless “scientific stuff”—measurements for the sake of measurements—and data that bear on im­portant hypotheses, concepts, and variables. In any case, accurately defining and rewarding all of the activities involved in testing hypotheses will be a major step forward.

12.