C Questioning and Model-Building (David J. Glass)24

QMB, the method championed by David Glass, also rejects the hypothesis. In QMB, a question comes first. Investigations progress by asking and answering more questions. Scientists reason inductively from experimental results and create models that explain the results.

While you might think that, with its emphasis on questions, QMB is very sim­ilar to Curiosity-Driven Science, nothing could be further from the truth. Let's look at Glass's definition of science to see why. Glass posits that “scientific re­search is the process of determining some property K about some thing X, to degree of accuracy sufficient for another person to confirm the property K.”²⁴ Note the requirement for accurate and reproducible measurement. For Glass, the world is sufficiently regular that inductive reasoning leads to models that we can count on to predict the future. Explanation is defined as successful prediction and “verification of [the model] is the explicit goal of the experiment.”

Science, in other words, is primarily an observational and descriptive activity; later efforts may determine cause-and-effect relationships, but the core value is accurate description. For Glass, scientists do experiments because they need to acquire data to answer one or more of seven types of questions: Does X exist? What does X look like? What is X made of? What does X do? What causes X to do K? By what mechanism does X do K? And, finally, what can perturb X to perform its function differently? In comparison with the flexible, unstructured Curiosity-Driven program, QMB is sharply defined and heavily prescriptive.

The question is the logical starting point for QMB science because scientists are in a state of ignorance. A crucial difference between QMB and Curiosity- Driven Science is the end product of an investigation which, for the Curiosity- Driven scientist, is the generation of new data and new ignorance and for the QMB program is another question or a model. For QMB, whichever has priority—question or model—depends on the circumstances. If your data are in­sufficient to support a model, pose a question; if you have enough data, construct a model. Once you have a model, you gather the “negatives and affirmatives,” the contradicting and supporting observations needed to verify or refine it.

Glass places high value on description, so he puts inductive reasoning—the process of extrapolating and generalizing from past experience—at the center of QMB. He believes that models arise from induction and, although he concedes that inductive reasoning does not allow for air-tight conclusions, he argues that the virtues of inductive reasoning outweigh its drawbacks. Moreover, he sees in­ductive reasoning as unavoidable: we necessarily rely on it to decide whether one model is a better predictor of the future than another. And, if things keep hap­pening the way a model predicts they should, the model gains “proof of verifi­cation.” This process depends on both induction and probability: when you’ve done a sufficient number of experiments, your data will reliably predict fu­ture outcomes. Glass admits that reliance on probability doesn’t solve Hume’s problem of induction (Chapter 1) but says that a probable outcome “shifts the burden to the critic.” He feels that there are “built-in preferences for particular kinds of models” and that a “workable” system should be satisfactory for science, even if it is not provable.

In QMB, the concepts of model and hypothesis are poles apart: (1) a model is based on data gathered during an investigation, rather than on the “prior un­proven assumptions” on which a hypothesis, in his view, must be based; (2) un­like the hypothesis, which is “established to be falsified,” the model must be held up for verification via inductive reasoning; and finally, (3) an unsuccessful model need not be scrapped, the way a falsified hypothesis must be, but can be used as a “starting point for suitably refined successors.”

Ideally, a scientist is an open-minded questioner.

To illustrate the importance of induction in biomedically relevant reasoning, Glass has us envision the ethical morass that wed be in if we abandoned induc­tive reasoning. Clinical trials are used to determine how much inductive power a particular medical treatment has. We must assume that the inductive conclusions from the study are valid if we're going to offer the clinical treatment to the public in good conscience. Glass acknowledges that the inductive power of conclusions drawn in biological sciences might fall short of the “absolute predictability” as­sociated with physical laws but asks where we'd be if past experience was irrele­vant to evaluating a treatment or a model. He considers whether falsification or verification should weigh more heavily in the evaluation and concludes that it is verification, which is provided by the inductive method and measured by induc­tive power, that is key.

10.C.1 Rejection of the Hypothesis

Glass stipulates that, for him, a hypothesis is not derived from data, that it is equivalent to the “fictitious and unrealistic” construct that was rejected by Francis Bacon and Isaac Newton. Glass cites Newton's definition of a hypothesis as “whatever is not derived from phenomena, an unproven premise, advanced without evidence, as a tentative explanation.” (See Box 10.1 for a discussion Newton's and Bacon's views.) Glass agrees and argues that a corollary follows: if an idea is well supported by data then it is, by definition, a model, not a hypo­thesis.

For Glass, hypotheses are aligned with “ideologies” and are dangerous be­cause they appeal to vanity. Furthermore, a scientist working with a hypothesis will “filter data through the lens of that hypothesis, rejecting contradicting evi­dence in favor of validating evidence.” In his favorite example, Glass suggests that an unenlightened scientist seeking to know the color of the sky would formu­late and test a narrow hypothesis such as “The sky is red.” To do so, the scientist would have to construct a special filtering device, a “redometer,” to distinguish between red and non-red colors and, as a consequence, would miss making any non-red observations. In another example, Glass considers a scientist testing the hypothesis that “caffeine increases blood pressure.” This person, he says, will try to set up conditions in which caffeine does increase blood pressure, rather than simply asking, “what is the effect of caffeine on blood pressure?” The very state­ment of the hypothesis compels the scientist to look for an increase, as opposed to a decrease, and establishes a filter that “forces methodology to determine an increase in particular.”

Additionally, Glass argues that a hypothesis sets up a “dysfunctional positive/ negative binary” that dictates that a scientist’s experimental objective is negation, whereas no one can doubt that hypothesis verification, not falsification, is the real goal of science. He wonders why anyone would spend hundreds of millions of dollars if their purpose were to disprove the hypothesis?

And insistence on a preexisting hypothesis stifles innovation. Glass believes that the National Institutes of Health (NIH) grant review system is failing because of a misguided reliance on hypotheses.

Finally, in case the reader might suppose that Karl Popper’s proposal to seek potentially falsifying evidence to test the validity of an idea would solve some the problems that Glass identifies, he waves away Popper’s program as “too prob­lematic,” inconsistent on philosophical grounds, and harmful to the process of discovery. And there is the problem of inconclusive falsification: if a hypo­thesis cannot be unambiguously falsified, then nothing has been gained. Glass concludes that hypotheses are “unhelpful, disadvantageous, inconsistent, and unworkable”; it is better for science to ask good questions.

10.C.2 Summary

Unlike the Curiosity-Driven approach, QMB enumerates detailed directions about how scientists should carry out investigations and stresses the value of pre­cise and reproducible measurements. QMB is philosophically compact: there are a few core principles and specific questions that guide its inquiries. Glass’s examples of scientific projects focus on detecting experimental regularities and deriving general principles to make models. He considers the process of model refinement as the reshaping of existing conceptual structures rather than aggres­sive attempt to probe their weaknesses or overthrow them. Glass sees science as a smooth, gradual, and increasingly accurate approach to truth; a vision that has as its goals practical measurement and workable solutions to existing problems, such as you’d find in applied science or industry.

10.C.3 Critique: QMB, Open-Ended Questioning

The model is at the heart of Glass's program, and his dismissal of the hypothesis in favor of the model rests on questionable assumptions about both concepts and their relationship.

What about the second part of Glass's characterization of the hypothesis, that it is “held up for falsification”? As I discussed in Chapter 2, this kind of comment stems from a misreading or misunderstanding of Popper. To review briefly, you conjecture a hypothesis as a true explanation for some phenomenon that you don't understand. You do not “hold it up for falsification” as if you're trying to prove that it is false; instead, you try to find out whether or not it is false. If your tests don't falsify it, then you continue to act as if it's true.

Look at Glass's test statement, “The sky is red.” As it stands, it's not a hypo­thesis, and that's part of the problem. Let's recycle it as a hypothesis: you are in a room late in the day, and you notice that light on the wall opposite the window has a warm, orange-red cast to it. You conjecture the hypothesis that “The sky is red,” to account for the color. Alternative hypotheses might include someone shining a light into your window or a fiery blaze in a nearby building. Your hy­pothesis predicts that, if you look out the window, you'll see that the sky is red and also that you won't see a light directed at your building or a fire next door. You look out the window to test your hypothesis. If the sky is indeed red, then you have learned a good deal more than that your hypothesis is “not falsified”— which is all that Glass will allow you. You have corroborated your hypothesis; that is, you've confirmed its prediction and falsified the predictions of alternative hypotheses.

Corroboration, once again, is not confirmation of your hypothesis about the sky, which could still be wrong; a neighbor might have been shining a red light into your window even if the sky was also red, but you do feel more confident in your explanation of the color on your wall.

The process of testing “The sky is red,” when it is a hypothesis that is based on data is not ridiculous or counterintuitive; it's the kind of thing we do all the time. Answering an open-ended question, such as “What color is the sky?” is a different matter because it does not suggest, or call for, a hypothesis; it is a matter for Discovery Science (Chapter 4). Keeping the distinctions between them in mind helps avoid confusion.

Unexpectedly, given the decidedly anti-hypothesis tenor of his book, at one point Glass suddenly acknowledges that hypotheses can be useful when you have adequate “prior knowledge of the system,” and he distinguishes between times when hypotheses are “practical” and when they are not. For instance, a hypothesis such as “gene 278 is a receptor tyrosine kinase” is “perfectly testable” and is within “the critical rationalist framework.” Now, he says, we are “back in the realm of useful hypotheses” (a surprise since we hadn't known that such realm existed for him). While in the realm, Glass sees hypotheses as scientists ordinarily do: as explanations whose purpose is to explain actual data. What changed, you wonder; can we arrange it so that we don't leave the realm again? Rather than explain, Glass, in a final wrenching twist of the argument, asserts that this association between the hypothesis and data is, nevertheless, a “problem for the critical rationalist framework” and segues back to his original position that, “strictly speaking,” hypotheses are not based on prior knowledge! It remains unclear why he fixates on a definition of hypothesis that no one has taken seri­ously for centuries.

In Chapter 2, I discussed the background information that scientists, in­cluding QMB scientists, have to rely on as a system of deep implicit hypotheses about nature. I'll return to this problem in Section 10.C. Before that, I want to ex­plore the distinction between models and hypotheses that is so crucial for QMB.

10.C.4 Models and Hypotheses: What's the Difference?

In QMB the model summarizes the results of an investigation, reveals relationships among experimental variables, and, at least implicitly, makes predictions about how it can be tested. I don't know how widely accepted his po­sition is. Stuart Firestein, for example, states that¹⁷ “A scientific model is more or less synonymous with a theory or a hypothesis” Glass firmly rejects this conclu­sion, however, Firestein's comment does raise the question of whether there are substantive distinctions between a hypothesis and a model. Glass proposes that models are summary statements (or diagrams, etc.) that are created to account for data presented in a scientific paper and therefore are necessarily found at its end, rather than at the beginning where, he implies, the hypothesis must appear.

Is it mandatory that a model be introduced only at the end of an investigation? Working scientists do not seem to think so. If a model is at all significant, some­body will challenge it and focus on it at the beginning of a new investigation. Indeed, in support of this surmise, I found (Chapter 9) that in 20 of 158 scientific reports the investigators introduced their paper by stating that their goal was to test a model; for comparison, 41 of158 explicitly said that they were testing a hy­pothesis. Hence, the evidence supports Firestein's intuition that scientists don't recognize a significant difference between models and hypotheses.

Why does any of this matter? One reason is that, in claiming that there is a conceptual difference between hypotheses and models, Glass's interpretation obscures the iterative, recursive nature of science. As Chapter 2 emphasizes, hypotheses are developed, tested, and, in response to experimental outcomes, modified (“recycled”), retested, etc. The model resulting from one investigation is the raw material that gets fed into the investigational hopper in the next cycle. It becomes the new hypothesis.

Glass's distinction between models and hypotheses includes the notion that a model is not subject to falsification. A major benefit of a model, allegedly, is its ability to make good predictions; when their predictions are inaccurate, models need “refinement,” or they can be “improved,” or “suitably revised.” Not falsified, though; only hypotheses are falsified in his scheme. Let's pause to assess this ex­traordinary position. For one thing, it easily accommodates the reasoning that sustained the Ptolemaic model of the heavens, which could always be “suitably revised” but not falsified. When their model made inaccurate predictions, the Ptolemaic astronomers refined it by adding more epicycles until the latest un­predicted wiggles of planetary motion were accounted for. Its inexhaustible flex­ibility made the Ptolemaic scheme unfalsifiable, and it has served as an archetype of bad or, to be fair, premodern, science. QMB holds the door open for such an approach.

But, Ptolemy to the side, I see no major distinction between a model that needs refinement and a hypothesis that has been falsified. You'd only need to re­fine a model if it failed in some way, and, when you correct its deficiencies, you replace it with a new version and get rid of the old one. Although “refinement” and “improvement” may sound kinder and less judgmental than “rejection,” trying to draw a meaningful distinction between “refinement” and “rejection” seems unlikely to get us anywhere. We could make a similar argument about “revising” and “rejecting” a hypothesis. I am reminded of the story of the axe that had been in one man's family for more than 200 years: the axe head had been replaced twice and the handle three times. He treasured it as the same axe that his great-great-great grandfather had used. However, hypotheses are not like axes; when you change part of a hypothesis, it becomes a new hypothesis. It's the same with models: a model that has been replaced by another is a model that has been falsified, even if you call it something else. What this means is that the results of testing don't distinguish between models and hypotheses.

QMB suggests that models can be verified, whereas hypotheses can only be falsified. This issue is not really about the properties of models and hypotheses per se; it gets back to the well-worn philosophical dispute about verification versus falsification (Chapter 1), which we don't need to rehash. Scientific models cannot be conclusively verified, and QMB does not claim that they can be; ac­cording to QMB, verification is always incomplete and conditional. This sensible position leads immediately to a conundrum: Glass rejects Popper's hypothesis testing program in part because falsification cannot be complete. We might ask, if partial or incomplete falsification is a big problem for Popper, why isn't in­complete verification a big problem for QMB? Or, if inconclusive verification is acceptable, why isn't inconclusive falsification acceptable? Again, it seems that what was intended to be a critical distinction between models and hypotheses doesn't stand up to close inspection.

Both models and hypotheses are explanatory interpretations that go beyond the data, involve simplifications and generalizations, and are the products of minds that seek to understand. The model advocated by Glass cannot escape whatever functional criticisms the hypothesis receives. Although hypotheses are singled out as sources of bias, QMB gives us no reason think that builders of models do not take the same pride in their creations or become as emotionally attached to them as do builders of hypotheses.

Once we put aside the antiquated definition of a hypothesis as a data-free fic­tional construct, we find no salient difference between models and hypotheses. We're left with a purely semantic one: if a simplified or idealized statement that summarizes and explains data and makes predictions appears at the end of a paper, it is a “model” by definition; if such a statement appears at the beginning of the paper, it is a “hypothesis,” also by definition. While you might argue that it would facilitate scientific communication if there were some fixed, agreed-on difference in meaning between model and hypothesis (I think this viewpoint has merit), I am afraid you'd face an uphill battle to get the scientific community to go along. As Firestein observes, scientists already use the terms interchangeably.

The model is one of the pillars of QMB, and another one is the principle of inductive power, which we'll turn to next.

10.C.5 Inductive Power

As I've noted repeatedly, philosophers have not been able to reach consensus about the utility of inductive reasoning (induction) to the quest for truth in sci­ence, and many of them concluded that it is of no value whatsoever. Induction continues to cause confusion, though, perhaps partly because the word has dif­ferent meanings. In one sense, induction is the name of a still-unexplained cog­nitive process that takes us from the experience of particular instances to the formation of general rules (later, we'll return to the question of whether this is a genuine “process”); I'll call this “cognitive induction.” “Induction” is the also name of an abstract philosophical method that is purported to enhance the truth value (or probable truth value) of scientific statements; this is “philosophical in­duction.” Problems can arise if we get the two ideas mixed up, and induction appears in both roles in QMB. There is no controversy about whether or not our brains continuously detect, report, and act on anticipated regularities in the world. We are always engaged in cognitive induction.

What about philosophical induction? Glass reviews David Hume’s criticism that the validity of induction rests on the unproven assumption that Nature is uniform. He does not consider it a kiss of death. Instead, while agreeing that in­duction cannot guarantee future results, he feels that a model’s ability to make successful predictions confers “inductive power” on it and that an accumulation of inductive power justifies confidence in the model’s future performance. What exactly is inductive power, though?

In essence, inductive power is a subj ective estimate of the odds that a model ac­curately represents the key elements of the data. We can imagine inductive power as an indication of how much to risk on particular outcomes—the best place to build a new road or the amount of life insurance to buy. Saying that a model has a lot of inductive power is equivalent to saying that somebody thinks it is right. This is not meant to trivialize the principle: there may be good reasons to think that a given model is right; experts may have weighed in with opinions based on high-quality evidence and rigorous statistics. Bayesian statistics (Chapter 6), for example, are well-suited to generating quantitative predictions that reflect the strength of hypotheses or models. Ultimately though, inductive power is still a temporary, probabilistic judgment that cannot satisfy the aspiration of science to seek the Truth about nature.

And, no matter how great its value, inductive power confers no protection against the unprecedented event, the rare or unpredictable anomaly, a “black swan”¹⁸ in the sense used by Nassim Taleb, an event that is totally unaccounted for by current models. If Taleb is right, history and science are more decisively determined by black swan events than by the regular, predictable occurrences that support inductively generated models, no matter how much inductive power they have.

10.