Testing the Virtual Model

This emphasis on computability and algorithms implies a correlated emphasis in formalization, on objectivity, but not necessary on “truth”. Simulating the past is just a way of increasing the explanatory power of historical explanatory models and not necessarily their “truth likeness”.

We never know for sure whether the generated computer model of historical transitions and changes actually describes what happened really in the past. It is important to take into account, however, that the mechanical generation of “hy­potheses” is no end in itself. A simulation can be “suggestive”, “imaginative”, “relevant”, “probable”, “plausible”, “credible” (Bankes et al. 2002; Garson 2009; Reynolds et al. 2013; Whitley 2016; Balzer 2015; Stettiner 2016). A generative model of the past that we belive existed is just a formal device to generate explanatory arguments that can be fitted to reality or not. As such, an “historical model” is just a deductive system as valid as its initial axioms. The only we can check is the deductive coherence, that is, that explanatory arguments are expres­sions generated by the system and hence coherent with the embedded assumptions. The degree to which that potential is realized is a function of the empirical validity of substantive models and the degree to which these theoretical ideas have been implemented clearly and accurately (Cederman 2002; Lustick and Miodownik 2009; Peeters and Romeijn 2016; Marwick 2016).

If virtual explanatory models cannot be tested, they can be explored. When exploring the resulting computable model of a causal trajectory of “events”, where each event is just a momentaneous state of the evolving system of agents, and all events within a trajectory constitute a “history”, we can generate large numbers of virtual histories by perturbing the chain of events randomly or introducing ran­domized adjustments in initial conditions.

Three methods of evaluating the validity of simulation models, over and above reliability, have been delineated by Taber and Timpone (1996):

• Outcome validity: demonstrating that outcomes in a simulation correspond to outcomes in the real world. Outcome validity corresponds to what can also be called “predictive validity” (Sterman 1984).

• Process validity: demonstrating that the process that leads to outcomes in a simulation corresponds to processes in the real world by calibrating initial parameters to empirically known historical data, in the sense proposed by Epstein (2006). Conversely, if the model omits real-world processes thought to be important in outcomes, the validity of model predictions is undermined even when those predictions have outcome validity. In some sense, it can also be considered a form of “predictive validity”.

• Internal validity: demonstrating that simulation software validly represents the process being modeled. Put another way, has the model been fully debugged so that a researcher can be sure that only explicit model assumptions are modeled without unintended effects due to software artifacts? This is similar to what others have called “structural validity”.

Turchin (2011) has advocated the use of historical experiments, meaning a planned comparison between predictions derived from two or more theories and data. In this way, we may focus on making predictions about the state of a certain variable for a certain past society, which is not known at the time when the pre­dictions are made.

Precise historical case studies offer an opportunity to examine the internal logic posited by a theory of transitions between different events. A good case study will trace the causal processes observed in situ and determine whether they are con­sistent with a specific theory or challenge it. Historical case studies frequently focus on a specific spatial and temporal scale, varying from small settlements in the past, to regional land-use changes. They are particularly well suited for testing theories that predict that some event or process will never occur. Many different methods can be used to observe the case, including archaeological data, historical docu­ments, ethnographical observations, remote sensing, surveys, censuses, interviews, etc. The various ways the system is measured may lead to some challenges when comparing cases with somewhat different observation procedures (Janssen and Ostrom 2006; Marwik 2016; Rubio-Campillo 2016; Heppenstall et al. 2016).

Therefore, empirical information, both qualitative and quantitative, can be used in a variety of ways. It can be used as input data to the computable model or as a means to falsify and test if not the model itself, its explanatory predictions. When historical data are used as an input, the focus might be to study a particular scenario, i.e., the proper historical circumstances from which the data is derived. By carefully calibrating start-up conditions to what is known from the past, crucial experiments can be designed to generate particular trajectories whose final states can be con­sidered as “predictions”, and then individually compared with what we know from the real past and measure its fitness. The more fitted are those latter states with equivalently dated historical data, the better the predictive power of the model. The revolutionary potential of this technique is associated with the fact that alternatively possible “futures” (or “histories”) can be produced by varying initial conditions or a specific parameter setting of interest or by subjecting the theoretically specified model to random perturbations.

1.2