E '1 he Variable Value of Reproducibility

'lhe preceding quote from the AAAS suggests that the appeal of reproducibility is not fixed; that it can vary with social context, for example. 'lhe fact that a man­date to pursue the goal of reproducibility may be “overly broad” suggests that we need to think about whether the value of reproducibility can vary according to the scientific context as well.

Usually, people who worry about the Reproducibility Crisis do not distin­guish between different kinds of science, even though reproducibility might not be uniformly vital in all cases. Recall that the present discussion of reproduci­bility was started by company scientists at Amgen and Bayer who wanted to de­velop new cancer therapies and couldn't repeat published results. These scientists needed specific biochemical targets to explore and hoped to pick up directly from where the published reports left off. When their replication efforts failed, they had nothing to work with. They were not trying to understand, say, the basic molecular biology of cancer, but to duplicate enough information to warrant spending time and money on drug development programs.

We've said (Chapter 4) that applied science is practical; it seeks to achieve tech­nological ends, whereas basic scientific research strives for a fundamental un­derstanding of nature, whether or not its practical applications are immediately obvious. The scientists at the commercial laboratories were doing applied sci­ence. In the terms introduced earlier, applied scientists are more interested in a report's data than its meaning. Basic science is more intently focused on meaning, and reproducibility of data is, in a sense, less critical for its advancement. Indeed, at times, it seems as if opposing groups avoid repeating each other's experiments directly, perhaps because pinpointing the reasons for failure of direct reproduci­bility is less interesting and cost-effective than getting on with new experiments. This suggestion probably seems counterintuitive to many readers, so I'll give an example in Box 7.2.

Divorcing specific results (data) from a bigger picture (meaning) happens frequently in basic science because higher value is placed on finding better explanations (hypotheses) than in merely making repeatable measurements.

I am not saying that reproducibility is unimportant, only that it is not the most important goal for science. We can look at the issue from a logical point of view: “True scientific findings will be reproducible” does not imply that “Reproducible scientific findings will be true.” If you argue otherwise, you are affirming the consequent, a fallacy that I mentioned in Chapter 1.

7.