C The Professional Scientist and the Hypothesis

Professional scientists hardly ever have the chance to get formal instruction re­garding the hypothesis and Scientific Method after they’ve earned their final ac­ademic degrees. There are, however, resources available to those individuals that might fill the void.

The most important federal agencies that fund research for biomedical scientists are the NIH and the NSF. Since scientific thinking is a major—maybe the major—point of a scientific grant application, you could imagine that NIH and NSF would be rich repositories of information about scientific thinking. Are they?

The 27 agencies that make up the NIH focus on particular diseases or disease- related areas, and they support all kinds of science (basic and applied, including clinical, hypothesis-based, Discovery Science, etc.) that are intended to benefit human health. There are NIH-wide policies regarding grant applications, which each agency supplements with its own specific rules. Biomedical scientists pay close attention to what NIH has to say and what information it offers.

Prompted in part by the recent hue and cry concerning reproducibility in sci­ence, rigor and transparency³³ have become major buzzwords at the NIH. The desire to enhance rigor and transparency has given rise to four guidelines meant to increase the reliability of scientific findings. Two of the four are fairly technical and won’t concern us. Instead, I’ll concentrate on those dealing with “rigor” and the “scientific premise.”

13. C.1 Rigor and Transparency

The policy on “rigorous experimental design for robust and unbiased results,” says that “Scientific rigor is the strict application of the scientific method.” The scientific premise is an innovation.³⁴ A grant application must have a strong premise, and a premise is not the same as a hypothesis.

Indeed, the word “hypothesis” does not appear anywhere in the policy, in the linked Frequently Asked Questions, or in an associated blog post.³⁵ Apparently the official policy of these agencies takes it for granted that scientists know about hypotheses. Fair enough, you say. The instructions are intended for professional working scientists who, it is presumed, only need to know how to apply for re­search grants, not how to think scientifically. Nevertheless, elsewhere on the NIH site, we learn that not all grantees do understand what a hypothesis is. An NIH blog writer³⁶ gives a dictionary definition of “hypothesis,” including a link to a definition suitable for “children,” adding that, “I am providing this as a public service. Recently I received a proposal to review from ‘scientists' [sic] whose hy­pothesis was actually a list of aims and methods. Perhaps it is wrong to assume that everyone with a science degree knows what this word means?”

Although the blog writer is incredulous, the responses to the “Hypothesis Overdrive” blog³⁷ that I discussed in the Introduction documents disagreements among scientists about what a hypothesis is. And, as someone who served on numerous NIH review panels, I can attest that a proposal betraying ignorance about the hypothesis was not a rare event. Thus, a fuller explication of the subject would not seem out of place.

Are matters like scientific thinking perhaps covered by the individual NIH divisions, rather than the umbrella website? To find out, I searched “hypothesis” on each of the 28 NIH subcomponent websites (27 institutes plus the Director's Office). The results were amazingly diverse.

Several sites gave a pro forma nod to the topic (e.g., “committed to new in­novative hypothesis-driven research”), however, the nature and quality of the information relevant to the hypothesis was spotty. For example, the primary references on the National Institute of General Medical Sciences (NIGMS) site are to the “Hypothesis-Overdrive” blog with its strongly anti-hypothesis bent. Only three, the National Institutes of: Allergy and Infectious Disease (NIAID); Mental Health (NIMH); and Drug Abuse (NIDA)³⁸ offered substantive advice on scientific thinking and writing. The most entertaining discussion that I found deals with “Stuffed Animal Science.”³⁹ which is intended to teach school children the basics of scientific thinking and is admirably straightforward.

Advice regarding hypothesis-based applications (“Application Missteps”) in grant preparation from NIAID⁴⁰ presents helpful examples of more and less “focused” hypotheses. “Unfocused” means that a strong central hypothesis is lacking; the hypothesis can be too broad (e.g., “inflammation is a key etio­logical component of autoimmune diseases”) or too descriptive (e.g., “we will evaluate changes in transcriptional signatures in the involved tissues following infection”). Unfortunately, the second example is not a hypothesis at all. The site implies that multiple hypotheses are desirable, but then notes that “Optimally, your experimental results should be able to prove or disprove your central hypo­thesis” implying that proof is possible and arguably perpetuating a misunder­standing of science.

The NIMH discussion of the hypothesis in grant writing is fairly extensive and is the only one that I found that gives the reader the invaluable information that a good hypothesis-based application will lay out a “win-win” outcome, meaning that, the site explains, even if the central hypothesis of the grant is disconfirmed, the data will constitute an important scientific advance. This is, of course, is the very essence of Karl Popper's program.

A search on “hypothesis” on the NSF website yielded nothing similar to the NIH's instructions for applicants. However, the NSF site is a font of data about the state of US science education, including international comparisons of science student achievements (we do not do well, although there are exten­uating circumstances). The site also tracks Americans' attitudes on a variety of topics, including global warming, medical research, and even, for a while, astrology.⁴¹

In summary, useful information on the NIH and NSF websites regarding hy­pothesis, prediction, testing, etc. is scattered and inconsistent. This is especially lamentable in the case of the NIH because of its position as the predominant funder of US biomedical research and, therefore, an oracular source of technical information for scientists and bioscientific knowledge for the general public.

13. C.2 How the NIH Can Help Educate Its Profession and Lay Clients About Scientific Thinking

1. Develop and make available acceptable NIH-wide definitions of hypo­thesis, prediction, test, etc.

2. Resolve the issues raised in the “Hypothesis Overdrive?” blog regarding NIH's position on hypothesis-driven research. Is NIH favorably disposed to hypotheses or not?

3. Provide instructions, for both grant applicants and reviewers, of exemplary hypothesis-based grants, including examples.

4. Elevate the worth of misnamed “negative data” by emphasizing how a well- designed, hypothesis-testing grant advances science by falsifying substan­tive hypotheses.

5. Establish guidelines that assist grant applicants and especially grant reviewers in distinguishing between hypothesis-based and non­hypothesis-based (e.g., Discovery Science) research and in evaluating them appropriately.

13.