Common Threats to Internal and External Validity
Conflict communication research poses many challenges to internal and external validity. For example, role-playing research has been examined for its problems relating to both internal and external validity; see Spencer (1978; see Duffy & Kavanagh, 1983; Geller, 1978, for related role-playing issues).
It seems that role-playing research is often dismissed as invalid because of an alleged lack of realism— mundane, experimental, or psychological. However, regarding research concerned with interpersonal violations, Gonzales, Pederson, Manning, and Wetter (1990) pointed out thatit is difficult to create comparable manifestations of accidental and volitional offenses in the laboratory, and judgment or role-playing studies may be the best vehicles for presenting subjects with numerous offenses that vary in terms of both the nature of the offense itself... and the objective severity of the consequence of the offense. (pp. 619-620)
Thus, research that employs role playing may be useful if the investigators can show that the study is realistic in the meanings relevant to the study.
Another potential threat to validity of conflict research is the use of college students as research subjects. For testing theoretical relationships, data gleaned from using college students may provide sufficient support for the theory, and if the theory is supported among college students, so the argument goes, the theory may be valid with other participants as well. This issue concerns both internal and external validity. There is evidence that the kinds of individuals who sign up for particular experiments are not representative even of the class of students who are sought to participate in such studies (Jackson, Procidano, & Cohen, 1989). Nonetheless, for the purposes of convenience sampling—as well as for theory testing—students are likely to continue to be used as participants.
Some scholars view external validity as a limitation of laboratory studies, but Berkowitz and Donnerstein (1982) discussed this matter at length and concluded that
experimental results may tell us something about the conduct of a broad range of people in natural situations even though the subjects and laboratory settings are not physically representative of this population or the real- world situations in which they are embedded. (p. 255; cf. Tebes, 2000)
Shapiro (2002), agreeing with this view, commented that external validity requires more than simple representativeness: “The surface representativeness of a study is usually not a good indicator of contribution to theory” (p. 491; see also Mook, 1983).