D Right Responses to Error: Lessons from the Airline Industry
I began this chapter with the question of why scientific authors often avoid stating their hypothesis when they have one. The discussion so far has centered on the paucity of trustworthy information about the hypothesis throughout the school years and on into the professional career.
Apart from formal sources of information, a myriad of other influences affecting scientists’ thinking that, collectively, constitute the values of the scientific community. In this last section, I want to consider a cultural force that affects how scientists behave; namely, the way in which the scientific community views falsification of hypotheses and the practice of labeling experiments that falsify hypotheses as “failures.”“Success hinges on how we react to failure.” So says Matthew Syed in Black Box Thinking: Why Most People Never Learn from Their Mistakes—But Some Do.42 Making mistakes is inevitable; the larger issue is how we respond to them. Syed contrasts the way that the airline industry responds to failure with that of clinical medicine. Errors in both sectors can be fatal. While it is difficult to make direct comparisons, the stark numbers are staggering: With tens of millions of flights worldwide each year, members of the International Air Transport Association had a fatality record of approximately 1 fatal accident per 8.3 million takeoffs in 2014.43 In US medical facilities alone, estimates range from 44,000 to more than 440,000 “premature deaths associated with preventable harm” each year, with an average estimate of approximately 210,00044 The US Centers for Disease Control (CDC) reports an annual total of about 149.5 million hospital visits (in- and outpatient combined) to US hospitals.45,46 Taking the estimate of 210,000 annual preventable hospital deaths, we get a rate of preventable deaths in hospitals, per 8.3 million visits, that is more than 13,600 times greater than the airline fatal accident rate.
Of course, people are more complex than airplanes, they come to hospitals because they are sick, and caring for sick patients is much harder than flying planes. Still, the conclusion that there is an unconscionably vast difference between airline safety and hospital safety seems inescapable. Syed argues that the ways in which each industry reacts to error is largely responsible for the gap. Flying errors trigger serious, systematic inquiries into their causes. The personnel involved are encouraged to come forward with information, and the black-box data is analyzed. The overriding purpose of the inquiry is to learn from the mistakes, not assign blame for what went wrong.In medicine, there is a pervasive fear of failure and the burden of unrealistic expectations. When perfection is expected by patients, lawyers, supervisors, and the doctors themselves, then admitting to a mistake amounts to acknowledging incompetence. Medical errors engender blame, often in the form of expensive and reputation-scarring lawsuits. Doctors develop habits of denial that keep them from recognizing and owning up to error. For Syed, this fosters a culture of “evasion” in medicine that is rooted in a dysfunctional attitude toward failure.
The good news is that change is possible. The airline industry was not always a paragon of virtue when it came to dealing with the nightmare of plane crashes. Years of investigation and analysis led to the conclusion that two factors are essential if large entities are to learn from their mistakes: there must be (1) a system that harnesses error for making progress and (2) a mindset that enables the system to flourish [emphasis added].
Falsification of a hypothesis is not a “mistake” or a “failure” in the senses of the mistakes and failures in the airline industry or medicine. But perception can be everything, and many critics—and scientists themselves—view a falsified a hypothesis as an indication that something went wrong, and this, I believe, enforces a reluctance to express hypotheses explicitly.
The good news here is that science has a system that, in principle, is set up to profit from its “mistakes.” The other factor, the mindset that sees error as opportunity for improvement, is harder to inculcate and gets less attention. The question is, “Is the current mindset of the scientific community open to making changes that can help science move forward?”You can avoid failure by creating goals that are so vague and loose that nobody, including you, can say whether you’ve achieved them or not. You can't learn from mistakes if you can’t identify them. We may be witnessing the embryogenesis of a culture of evasion in the lack of a stated hypothesis in published research, in the overt opposition to hypothesis-based science voiced by critics, in the disorganized approach to teaching students about scientific thinking, and in the diminished status of the hypothesis in the eyes of granting agencies, such as the NIH. We should do our utmost to stop a culture of evasion from developing, and one big step toward stopping its development is to promote regard for the Scientific Method and its most valuable tool, the scientific hypothesis.
13.