Robustness, Intersubjective Reproducibility, and Scientific Realism

Although the distinction between different notions of robustness is necessary in order to avoid confusion (cf. Woodward 2006), we should not run into the opposite error of neglecting important similarities between these different kinds of robustness.

As many authors already emphasized-from Frege to Poincare, from Wittgenstein to Popper-, a particular or single event of perceptual awareness (for instance, my perception of a blank sheet of paper lying before me) is not only absolutely certain, but also unavoidably subjective and private, because it is not accessible to any other person. As such, it has no right of citizenship either in science or in empirical knowledge in general. Such a perception belongs to a subject, and not to an object, and for this reason it is not intersubjectively testable in principle. As for example Popper rightly noticed, we do not take even our own observations seriously, if they are not in principle intersubjectively testable (Popper 1959[2002], 45). On reflec­tion, to regard such a perception as the property of some empirical object is much like conceiving it as a mere hallucination. If someone sees a lion in a room of the house, s/he would perhaps look again and/or ask someone to test whether s/he is seeing the same thing because there is something decidedly strange, if not impossible, in this perception. If no one could find any trace of the lion later on, s/he would know that what s/he had seen was a hallucination, no matter how frightening it was.

If, according to the interpretation of Hacking's realism presented in the pre­ceding section, we consider intersubjective reproducibility from the point of view of a technical-experimental conception of knowledge, the present point may be for­mulated by saying that we do not believe that our own observations refer to something real, if they are not in principle reproducible and therefore intersub­jectively testable, in relation to the interactions of human beings with the sur­rounding world.^[55] From this point of view, what makes robustness-as-stability so important in science is the fact that it is intimately connected with intersubjectively testable reproducibility: intersubjectively constant and stable reproducibility is, in the last analysis, the most valuable clue to the independent existence of particular empirical objects (and their properties) and therefore of the truth of propositions which refer to them.

But what about robustness-as-consilience? Is it also connected with intersub- jective reproducibility as an essential pillar of scientific experimentation?

There is at least an important sense in which robustness-as-consilience is a valuable clue to reality as intimately connected with intersubjective reproducibility, a sense that will put us in the position to appreciate the limits of the objections mentioned at the end of Sect. 2. In this section I will focus on the first objection; I shall postpone the discussion of the second one until the next section.

The first objection was that robustness-as-consilience reasoning does not provide any autonomous epistemic warrant if one lacks minimally reliable observational data or procedures; for this reason, the robustness requirement must be integrated by a requirement of “minimal reliability” (Hudson 2014, 18).

It must be conceded to Hudson that reproducibility of robustness-as-consilience is reliable only if it is already present, at least to some extent, at the level of robustness-as-stability.

Take for example sense perceptions in everyday life. On the one hand, robustness-as-stability must be already present at the level of everyday life, when for example we see again the same thing or hear again the same noise in order to increase the reliability of the judgments we have made. If a certain intersubjective reproducibility of what is perceived by the sense organs, taken separately, were not presupposed from the beginning, adding a second reproducibility to increase the reliability of the first one would be pointless.

On the other hand, however, if a certain intersubjective reproducibility of robustness-as-consilience—that is, of the consistency with which different pieces of evidence point to the same conclusion—, were not given, improving the degree of reproducibility of each piece of evidence would be equally pointless. We feel confident in the reproducibility of the results of any of our interventions on reality (robustness-as-stability) if the different interventions are stably consilient, that is, if robustness-as-consilience is itself intersubjectively reproducible to a sufficient degree—sufficient for our purposes. The fact that household objects such as flour, sugar, milk, eggs and currants can be handled with high reproducibility would not be a ‘fact' (strictly speaking, they would be absolutely useless to us!) if they were not regarded as stably consilient, that is, as coherently placed into the whole of our (in this case domestic) life.

This point may also be aptly illustrated by a brief analysis of the notion of scientific experiment, which somewhat reflects the unity and distinction of the two senses of robustness for which we have been arguing. For this purpose, we may adopt Mach's definition of experiment. It would lead me too far away from my present purpose to discuss Mach's definition. I shall simply adopt it as one which seems to me to provide a viable definition of experiment.

As Mach wrote, a scientific experiment is based on the ‘method of variation' (Methode der Variation), whereby some variables are systematically modified to establish which relation of dependence, if any, holds between them:

The basic method of experiment is the method of variation. If every element could be varied by itself alone, it would be a relatively easy matter: a systematic procedure would soon reveal the existing dependences. However, elements usually hang together by groups, some can be varied only along with others: each element is usually influenced by several others and in different ways. Thus we have to combine variations, and with an increasing number of elements the number of combinations to be tested by means of experiment grows so rapidly (a simple calculation shows this), that a systematic treatment of the problem becomes increasingly difficult and in the end practically impossible. (Mach 1905[1976]: 202-203, Engl. Transl. p. 149, translation slightly modified)

As far as robustness-as-stability is concerned, its importance for scientific experiment is evident from the fact that a reproducible procedure that reveals “the existing dependences” is part of what makes a good experiment, that is, an experiment able to deliver sound conclusions. From Galilei's experiments on the vacuum to experiments in quantum physics—where we are able to establish rela­tions, not between events, but between the frequency distributions of observed events (on robustness in quantum physics, cf.

However, in so doing, can the experimenter proceed without any reference at all to robustness-as-consilience? It seems to me that it does not, for the following reasons.

Mach's method of variation holds only on the understood condition that no disturbing force intervenes, and in order to assume this with a degree of certainty that is sufficient for his/her purposes, the experimenter might find himself obliged to go beyond the limited conception of robustness-as-stability here considered. As Mach noticed, usually the scientist can vary some elements only along with others because they interact with several others and in different ways. For this reason, we have to “combine variations” (Mach 1905[1976], 203, Engl. Transl. 149), that is to say, we have to combine and to compare different experimental interventions on different experimental apparatuses and laboratories in order to distinguish, in a system of correlated variables, causal from non-causal relationships, and therefore causally dependent variables from causally independent ones.

Simply varying conditions in the same experimental set-up seems clearly to serve the purpose of increasing robustness-as-stability, but varying the experimental apparatus and/or the laboratory is clearly relevant to the aim of increasing the degree of robustness-as-consilience, in the hope that facts already experimentally established (and for this reason already relatively independent) are found to be reproducibly consistent with one another. This follows from the fact that natural laws are in principle to be concretely exemplified in the functioning of ‘technical machines' or technical apparatuses built and mastered by the scientists, in which that law is present and operates in a controllable form (for instance, a pendulum for the laws of pendulum motion; determinate measuring instruments for the laws of free fall etc.).^[56] But if natural laws can only exist as exemplified in concrete technical apparatuses or measuring instruments in which that laws operate in a reproducible form, a change of experimental apparatus and/or laboratory involves a change in the law, and, more precisely, if this leads to success in reproducing the same outcome, it involves an extension of the old law to a new field of phenomena.

We may illustrate the point from a slightly different point of view. As Gooding rightly pointed out,

[e]ach test of what may be called the same theory in different laboratories will invoke different background knowledge, enabling assumptions, local resources, and competences [...] Experimentation is largely about identifying just the assumptions that matter in the world as engaged in that particular laboratory. (Gooding 1992, 69).

In order to obtain a well-reproducible and exemplary experiment, according to Gooding, scientists have to eliminate “nature's recalcitrances”, which, on the one hand, impede them in the search after generality, but, on the other hand, indicate a discrepancy between theory and practice and help identify those aspects of the world that are peculiar to contingencies about a particular laboratory at a particular time (cf. Gooding 1992, 69).

Now, it should be obvious on a little reflection that, if scientists can reproduce an experimental result in different laboratories—with different background knowledge, enabling assumptions, local resources, and competences-, they, in principle, have already overstepped the limits of robustness-as-stability as usually understood.

Admittedly, there is a huge difference between, on the one hand, the generali­sation and extension of a law beyond the limiting conditions of one laboratory, and, on the other hand, Newton's physics, which unified, to use Whewell's words, “remote and unconnected quarters” (Whewell [1840] 1847, Vol. 2, 65), such as Kepler's Laws, the fact that the planets would slightly disturb one another's motions, the perturbations of the moon and planets by the sun and by each other, the fact of the precession of the equinoxes, etc. But this difference, however great it may be, is only of degree, and not a conceptual one. The extensions of important experiments to different laboratories and the unification of disparate phenomena differ only in degree and not in kind.

Finally, the same result may be reached by still another consideration. Polanyi pointed out that reproducibility may depend “on the presence of an unknown and uncontrollable factor which comes and goes in periods of months or years and may vary from one place to another.” (Polanyi 1946, 79.) On the one hand, this does not prove anything against our preceding argument, according to which intersubjective reproducibility can be regarded as the most valuable clue to the existence of independent empirical contents if it is taken in relation to the interactions of human beings with the surrounding world. From this point of view, Polanyi's remark only means that the ultimate criterion of claims about experience can only be experience, and exactly for this reason there can be no final certainty in experimental knowl­edge any more than in any other field of human life. We can never attain absolute certainty, since the degree of certainty to which we may come always depends upon the extent to which we are able to make our experience intersubjectively repro­ducible, and more precisely upon the extent that is sufficient for our practical purposes.

On the other hand, however, it is exactly the possibility of conceiving any experimental reproducibility as a mere coincidence which involves the impossi­bility of confining intersubjective reproducibility to robustness-as-stability. Again, robustness-as-stability is to be supplemented with some degree of robustness-as- consilience.

For this reason, to suggest, as Hudson does, that it is always better to attempt to improve the reliability of a single procedure than to examine the convergence on the same result of different procedures (see e.g. Hudson 2014, 7-8), is not only a hasty and sweeping generalisation, but it is also wrong. In fact, contrary to Hudson (cf. Hudson 2014, 145, with reference to the existence of dark matter), one has to recognize that improving the reliability of a single procedure is hardly decisive or sufficient if taken in isolation. In this sense, Soler is right in complaining that Hudson only shows the deficiencies of what she aptly calls “blind robustness” (Soler 2014, 210), that is, a form of robustness that pretends to be the only method of interpreting intersubjective reproducibility as the distinctive trait of that which we believe to be real (or, what comes to the same thing, of that which we, with a greater or less degree of certainty, assert to be true).

Therefore, it is no fatal objection that robustness-as-consilience reasoning cannot be credited with epistemic warrant if one lacks a minimally reliable observational procedure (which is an expression of robustness-as-stability), since what is important in science is not robustness-as-consilience in itself, but only the synergy (or, if you prefer, the robustness of higher order) between robustness-as-consilience and robustness-as-stability, which are two complementary aspects of the same technical-experimental reproducibility in its more general sense. Robustness-as-stability, if understood as an aspect of experimental reproducibility, is not an alternative to robustness-as-consilience, but it is one of the most important requirements that must be met by a piece of evidence before entering into relations of consilience, which remains an essential integration in the search for more reliable knowledge. What Hudson believes to be “a key part” of his critique of robustness (Hudson 2014, 6), is only a reason for maintaining that it is not sufficient to distinguish between two senses of robustness; it is also necessary to connect them with one another.

Thus, robustness as stability of results is only one aspect of a relation, but an aspect that, in the concrete, must be, at least in principle, inseparably bound up with the other aspect, that is, robustness as consilience of different pieces of evidence. For this reason, scientists look for interdependence and mutual growth and devel­opment, that is, for synergy, of the two elements upon which the aforementioned senses of robustness are based.

To sum up, robustness-as-stability and robustness-as-consilience are two side of the same experimental and intersubjective reproducibility, which, in this sense, may be plausibly presented as the core of robustness in science. In other words, in principle intersubjective-experimental reproducibility is the most general condition of robustness, which applies to both senses of robustness with which we have been dealing.

5