C Is the Hypothesis Especially Likely to Elicit Biases and Spawn Fads?
The topic of “bias” is multifaceted, so I've broken it down. In this section I focus on the broad general notion of bias and put off most of the complexities for the next chapter, where we'll get into a few of the specific biases that are currently the source of concern regarding the Reproducibility Crisis.
The main reason to split up the discussion is that there are two opposed schools of thought regarding bias, and, to understand them, we'll have to explore the concept of rationality, and rationality itself is complex. We'll put that subject off until Chapter 12. At the moment, we'll take bias to be an intellectual leaning that may or may not be justified and look at it in relationship to the hypothesis. Critics going back to Frances Bacon have attacked the hypothesis for being unusually prone to evoking bias in the minds of hypothesis-holders. But what does that mean? Does hypothesis-based thinking inherently induce bias? If people, including scientists, form biases, does that mean that they shouldn't develop hypotheses? Is biased thinking always bad practice?11. C.1 The Hypothesis and Bias
When critics refer to the problem of bias in science, they often mean something like “unjustified” preference for one's own ideas and the neglect or disparagement of alternatives. The term “unjustified” is slippery, though. If a scientist defends her conceptual model that just seems right to her or is aesthetically appealing28,29 and is not in conflict with data, is she wrong? Richard Feynman was renowned for vigorously defending his unique ideas even when he had trouble communicating them to other physicists.30 The two-time Nobel Prize winner in physics, John Bardeen, was once stymied during an oral examination when he couldn't describe how he'd come up with the correct answer to a challenging question he'd been asked.31 Neither Feynman nor Bardeen could always justify their conclusions, but how can we understand their strong devotion to their ideas except as a kind of bias? Biases may arise from our unconscious minds and cause us to do or think things that to others might appear indefensible although the defense may be merely inexpressible.
The decisions scientists make as to which experimental avenues to follow can represent biases as well. In their study of synaptic transmission (Chapter 2), Bernard Katz and Paul Fatt chose to focus on one phenomenon in their data and to turn a blind eye to another one. It proved to be a brilliant choice, but they couldn't have fully justified it based on what they knew at the time. In countless cases, scientific “intuition”—many times a polite synonym for “bias”—has led to crucial discoveries, which means that a priori justifiability cannot be a prerequisite for good science. On the contrary, competitive bias that prompts you to defend your hypothesis may benefit science, just as competition improves commercial businesses. Good ideas are not made worse because someone vigorously defends them. Neither all biases nor all competitions are good, but it's an error not to distinguish the good from the bad.
Is it true, as some members of the anti-hypothesis contingent imply, that if you're biased in favor of your hypothesis that you'll somehow profit unfairly? It seems unlikely that this is a major problem. There is no doubt that a dramatic new hypothesis can attract a lot of attention and short-term benefits; however, the long-term advantages should not be exaggerated. Despite the vaunted status of priority of discovery in science, it is rightly said that the major credit goes to her who does it best, not to her who does it first. Moreover, if your alluring, though biased, hypothesis is falsified, then the falsifier reaps more reward than you do. In other words, you're unlikely to enjoy lasting benefits from putting forward a hypothesis that is founded mainly on bias. Science aims to get things right; it claims to be correctible, not infallible. One more point here: even when scientific bad behavior does show up as bias in favor of a hypothesis, as it surely can at times, blaming hypothesis-based science per se makes no sense. Correctly deployed hypotheses do not generate bad science; scientists generate bad science.
Perhaps a misunderstanding of the rewards of science amplifies the perceived dangers of bias. You don't earn respect in science for the same traits that would win you respect in other areas of civil life. We've all heard the politician who brags about his resistance to changing his mind, “if I had it to do all over again, Id do it exactly the same way,” etc. In the words of one prominent former government official, “Admitting doubt or mistakes is career suicide”32 Yet essential requirements of the job description of a scientist are a willingness and an ability to change your mind when the facts demand it.
And scientists are admired for different accomplishments than certain other professionals. A good criminal lawyer who gets a guilty client off scot-free is respected, not looked down on. Professional sports teams do not forfeit victories just because a referee missed a call. Refereeing, flaws and all, is considered part of the game. The linemen in professional American football are said commit illegal “holding” infractions on essentially every play and are lionized for their cagi- ness. There are occasional bright ethical lines in sports: if an athlete's cheating threatens the sport's fundamental integrity (say he gets caught using illegal performance-enhancing drugs), then he can be sanctioned. (Lance Armstrong,33 the disgraced professional bicycle rider, is the poster boy for high-level sports cheating.) Nevertheless, much bad behavior in sports is praised or overlooked, but not punished.
Scientific fraud represents the nth degree of bias toward one's ideas. If being biased paid dividends, then Jan Henrik Schoen34 would be a fantastic success story. Schoen managed to get many fake physics papers into top journals before he was caught, publicly humiliated, and fired. He was not rewarded. Inflexible and unjustified bias in science is a recipe for disaster, or at least for being thought stupid. Scientists who don't change their minds when the evidence goes against them eventually suffer the consequences.
“Bias” is a catch-all term for a multitude of behaviors—good, neutral, and bad—which is why no simple fix will make it go away. Once your hypothesis is public, any biases you might have are open for discussion and correction. Besides being careful and self-critical, applying the lessons of hypothesis-based thinking is the best way to avoid trouble.
11. C.2 The Hypothesis and Scientific Fads
Are hypotheses especially likely to create “bubbles of interest and attention”35 (which to save space I'll call “fads”)? Is there something about a hypothesis that causes otherwise mature, level-headed scientists to behave like the frantic, greedy investors who got caught up in the white-hot housing market of the early 2000s and went bust when the bubble began to burst in 2007? Like biases, fads are often the objects of derision that may not be entirely deserved, and we can ask the question we asked of biases: Are hypotheses bad because they attract followers? It's a reasonable question because science can, no doubt, be faddish. At times, if you're not using the latest technique or into the most fashionable avenues of research, you can feel sidelined. Still, fads, like bias, have their positive and negative sides. Bright young scientists may be attracted to a branch of science because it is flashy—and in rapidly expanding areas of research the funding is good and scientists do need to make names for themselves. A wave of interest in a scientifically and socially important problem can propel needed research. Autism, schizophrenia, Alzheimer’s disease, and drug addiction are currently hot topics in neuroscience, and progress in solving any one of them would be a huge boon for society. Or, take the case of Ebola. Over a 20-year span (1990— 2010), a total of 783 scientific articles with “Ebola” in their titles came up in an online search of PubMed (i.e., about 39 a year). As the epidemic surged in parts of Africa, so did the number of studies; in 2014 alone, 830 studies of Ebola were published, which probably contributed to its eventual taming (although social and political factors played a key role36).
In the past, concern about HIV/AIDS triggered an avalanche of research that resulted in HIV being commuted from a death sentence to a treatable disease. If societal interest and scientific incentives create fads, then scientists will follow fads, and that’s not necessarily a bad thing.Furthermore, the dramatic announcement of a new finding can speed up the rate at which it is replicated or falsified, which might ordinarily take many years. In 1989, a report37 that clean nuclear energy could be generated (“cold fusion”) in a desktop device using common laboratory materials and equipment got a bandwagon rolling. If true, the finding would have revolutionized physics and probably altered the course of civilization, with its promise of cheap, limitless energy to everyone on the planet. The report briefly rocked physics and precipitated concerted attempts to test the hypothesis, all of which hastened the virtual extinction of the field when it was not corroborated. A similar flare-up in biology accompanied the “discovery” of bacteria that, supposedly, could live on arsenic instead of phosphorous.38 If arsenic could be incorporated into DNA and RNA, then phosphorus would not be the mandatory molecular building block of life that everyone thought it was, and the textbooks would have needed rewriting.39 This finding, too, soon turned out to be a dud. The hypotheses of cold fusion and arsenic-loving bacteria created fads that, because of the intense interest they generated, fizzled, which allowed the scientific record to be quickly corrected.
Naturally, scientific herd behavior is not always praiseworthy. It can deflect young scientists from more productive research paths, delay new and important discoveries, siphon off research funds to narrow specialties, and leave solid, but less-nimble, laboratories in the lurch when scientific winds suddenly shift. The sociology of science, including a tendency to follow fads, is too complex and multidimensional to allow for simplistic generalizations.
The pressing question here is, “What do fads have to do with the hypothesis?” Is there evidence that, for example, Curiosity-Driven Science, or questioning and model building (Chapter 10) are less likely to trigger fads? Has anyone made the case that opening of a new line of research by a Discovery Science project is immune from inspiring groups of imitators to follow suit (i.e. from giving rise to a fad)? If not, then why blame the hypothesis for fads? Conversely, if researchers were drawn to a particular area of hypothesis-based science because they could readily understand it, appreciate its significance, and see how to test it, should we fault the hypothesis? Or credit it for fostering good scientific conduct?
11.