Causal Analysis, Internal Validity, and External Validity
Studies that test models or theories seek to establish the causes and effects at work. When several variables are involved, we may also try to know which variables may be mediators (intervening variables) and which may be moderators (interacting variables).
The difficulty involved in establishing causality arises because of the three canons of causality that must be met. The first canon involves c ovariation, which means that the predictor variable and the predicted variable have a relationship that is something other than a flat line: As the level of the predictor variable changes, the level of the predicted variable changes systematically. The second canon concerns c orrect temporal order, which means that changes in the predicted variable do not occur before changes in the predictor variable. The third canon refers to nonspuriousness, which means that there is no other variable that can fully account for the covariation between the predicted and predictor variables.Internal validity is concerned with whether the causal conclusions that an investigator may make are nonspurious. When our claims about causality are false, the study lacks internal validity. (Note that the discussion here differs from that of Cook & Campbell, 1979, with regard to distinctions and terminology.) The catalog for sources of invalidity—other explanations for the predicted outcome than the variable predicted to cause it—was begun by Campbell and Stanley (1963) and continued by Cook and Campbell (1979) and Campbell and Russo (1999). The list of possible sources of internal invalidity includes history (events that occur between the observation of the independent variable and the dependent variable), selection (a variable that determines who receives which level of the independent variable because of enrollment not under the control of the investigator), m ortality (a variable that determines who receives which level of the independent variable because of disen- rollment not under the control of the investigator), regression (effects due to the study’s focus on those who are extreme on the independent variable or a variable correlated with the independent variable), testing (the effect on the dependent variable due to being pretested on it), and instrument decay (systematic change in the measuring instrument; the instrument could refer to coders, physiological measures, or self-report scales).
External validity is concerned with our ability to generalize the results of a study to other populations, times, settings, and manipulations or measures of the variables.
To generalize, we first need something to generalize: We must have internally valid results. The usual criticism about laboratory experiments is that they lack external validity because they are conducted in artificial settings, and participants know that they are being studied. Aronson and Carlsmith (1968) and Aronson, Wilson, and Akert (1994) introduced three concepts to clarify these issues: (1) mundane realism, the extent to which the activities in the experiment are similar to activities that the participant generally engages in outside of the experimental setting; (2) experimental realism, the extent to which the participant takes the experiment seriously and feels involved in it; and (3) psychological realism, the extent to which the participant uses the same psychological processes within the experimental setting that the participant generally uses outside of this setting.