ADVANTAGES AND DISADVANTAGES OF EXPERIMENTATION
As mentioned earlier, the alternative to doing an experiment is to do a correlational (also called observational) study, in which all the variables are measured. For example, consider the hypothesis tested in the horn-honking experiment described above, that emotions competing with anger will inhibit aggression by people who have been provoked.
How might it have been tested in a correlational study? The researcher (Robert Baron) might have asked participants to remember the most recent incident in which they were frustrated or provoked by another person. To measure the independent variable, he might have asked them about additional experiences they were having at the same time and classified these experiences into those that would produce competing emotions and those that would not. To measure the dependent variable, he might have asked them how they reacted to the provocation and graded the aggressiveness of that reaction on a scale from 1 to 7. The hypothesis would have been supported if aggressiveness had been lower when competing emotions were present than when they were absent.There are both advantages and disadvantages of doing experiments rather than correlational studies. This means that both kinds of study have their place in research on social conflict, depending on the circumstances and the goals of the research.
Advantages of experimentation
There are at least five advantages of doing experiments. Some are so compelling that serious consideration should be given to using this method in most research.
Creating novel conditions
One advantage is that experiments allow the study of conditions that do not ordinarily occur. For example, in the hypothetical correlational study just described, it might be hard to find people who were experiencing competing emotions at the time they were frustrated or provoked. But Baron easily produced such emotions in his experiment.
Similarly, experiments are necessary to test the effectiveness of conflict resolution techniques that are not yet in use. Thus, Conlon, et al. (2002) evaluated arb-med, a novel kind of third-party procedure in which an arbitrator makes a sealed decision about a controversy, which then goes into mediation. If the mediation fails, the arbitrator’s decision is opened and becomes binding on the disputants. In an experiment involving a simulated merger negotiation between two companies, this procedure was compared with med-arb, a common procedure in which arbitration occurs only if mediation fails. Arb-med turned out to be superior, with more disputants settling in the mediation phase and settlements involving larger joint benefit.Establishing cause and effect
The second advantage is that experiments make it easier to distinguish cause and effect among the variables in the study. For example, in the hypothetical correlational study just described, there could have been ambiguity about the causal direction between the two variables that were measured. Aresult supporting the hypothesis could have been due to the impact of competing emotions on aggression. But it also could have been due to the impact of aggression on memories about competing emotions. In other words, heavy anger and aggression might have caused participants to lose sight of other concurrent experiences, including competing emotions. Ambiguity about causal direction was not a problem in the horn-honking experiment, because the presence or absence of competing emotions was produced by the researcher and hence could not have been influenced by the participants’ levels of aggression.
The following is a broader statement of the advantage of experiments over correlational studies for assessing cause and effect: when a study of any kind shows a relationship (covariation) between two variables, X and Y, there are four possible explanations: (1) X (or an associated variable) influences (has a causal impact on) Y; (2) Y influences X; (3) some third common factor, Z, influences both X and Y, producing a ‘spurious’ relationship between them; (4) the relationship between X and Y is due to chance.
Statistical tests of significance allow us to rule out explanation 4 at an acceptable level of confidence. Beyond that, it is a matter of reasoning, weighing the plausibility of each of the other three explanations. If the study is an experiment and X is a manipulated variable, we can rule out explanations 2 and 3, by arguing that the investigator is the source of variation in X and hence neither Y nor Z can have influenced X. This reasoning leaves explanation 1 as the only plausible account - that the relationship was produced by X (or an associated variable) having a causal impact on Y.In correlational studies, it is sometimes possible to explore cause and effect by means of path analysis or causal modeling, but these methods seldom allow watertight conclusions. Only experiments definitively sort out cause and effect.
Reducing confounding
Another important advantage of experiments is that they allow purification of independent variables by reducing the amount of confounding. When we manipulate or measure a variable, X, we are always inadvertently manipulating or measuring a number of other ‘confounded’ variables (A, B, C, D, E, etc.) that covary with X. If an X-Y relationship is shown in our data, it could be due to a causal relationship between one of these confounded variables and Y rather than between X and Y. Variables such as X can be manipulated with much more precision than they can be measured. Hence, the number of confounded variables is greatly reduced in most experiments, and it is easier to pinpoint the precise source of any changes that are found in the dependent variable(s).
This purification can be seen in the hornhonking study. The researcher held constant across the four conditions many variables that might otherwise have been confounded with the intended variable. Thus, the procedure was exactly the same and the same intersection and confederates were used in all four conditions. Furthermore, he randomly assigned participants to conditions, making it unlikely that the conditions would differ in the type of participant assigned to them.
Confounds of this kind are much greater in correlational studies, where circumstances and participant characteristics often differ substantially across conditions.Experiments are never completely successful in eliminating confounds. For example, in the amusement condition of the hornhonking study, though participants did laugh at the clown garb, it is possible that other psychological states produced the low level of aggression. Examples are surprise, disbelief, and psychological distancing, all plausible confounds of amusement. The solution to this residual problem of confounding is to do further experiments in which the variable(s) in question are manipulated in other ways that involve different confounds (Carnevale and De Dreu, 2005; McDermott, 2006).
Controlling for random variation
Another great advantage of experimentation is that it is possible to treat all participants in a given condition exactly the same way. This reduces random variation - differences between participants in the way they behave in a given condition - increasing the chances of reaching statistical significance if one is testing a valid hypothesis. Random variation is typically much greater in correlational studies, which means that a much larger number of participants must be used to reach statistical significance.
Assessing process
In experiments, as opposed to correlational studies, researchers are usually closer to the process that generated the relationship between the independent and dependent variables. This means that it is usually easier for them to map out that process.
Attention to process is illustrated by a laboratory experiment performed by Cohen et al. (1996) to test another version of the culture-of-honor hypothesis, that Southern men who are insulted will exhibit more dominance than men from other parts of the country. Participants were white college students from the South and the North. The events in the experiment occurred in the following order: first, the participant was asked to fill out a questionnaire and take it to a table at the end of a long, narrow hall.
As he went down the hall, he had to brush by a male confederate who was working at a filing cabinet. In the insult condition, the confederate slammed in the filing drawer, bumped the participant, and called him an ‘asshole’. In the control condition, the confederate did nothing.Two measures of dominance were then taken for all participants. First, as the participant continued back down the hall, another confederate, a 250-pound male, came around a corner and walked rapidly toward the participant in what amounted to a game of chicken. Dominance was measured by how close the participant came to the confederate before swerving. Then the participant shook hands with a third male confederate who had seen him being bumped and was supposed to evaluate him. The second measure of dominance was the firmness of this handshake. As hypothesized, Southerners who had been insulted swerved later in the game of chicken and shook hands more firmly than in the control condition, but the insult did not affect how the Northerners behaved.
Three process measures were used in this and a comparable experiment. At the end of the experiment, participants were asked to guess what the third confederate thought of them on various dimensions. The insult caused Southerners to report that they looked weak and cowardly, but it did not have the same effect on Northerners. This suggests that the insult may have caused the Southerners to feel that their masculine reputation had been damaged and, hence, that dominance was necessary to repair this damage. In addition, saliva samples were gathered from the participants at the beginning and end of the experiment. Chemical assays showed sharply increased levels of cortisol and testosterone in the insulted Southerners but not in the control Southerners or the two groups of Northerners. The cortisol results mean that the insulted Southerners were experiencing an unusual level of stress, and the testosterone results mean that their hormones were preparing them for dominance or aggressive behavior.
These three process findings give us insight into the psychological and physiological mechanisms linking the independent and dependent variables.Advantages of correlational studies
Some variables are very difficult or impossible to manipulate and hence must be examined by means of correlational studies, for example, the behavior of the heavenly bodies or of nation states. If we want to do experiments on these phenomena, we are forced to employ simulations, which are always simplifications of the real thing (see next). It is also not possible to manipulate participant characteristics, such as sex, age, and race. In addition, there are ethical objections to manipulating some variables. Even if we were able to create serious marital quarrels to see how they affect the offspring, we should not try to do so. To stay within ethical boundaries, we must measure such variables.
Correlational designs must also be used when one wants to look at the interrelations among a large number of variables, since only a few variables can be manipulated in an experiment. A case in point is a survey study done in several Crimean towns and villages by Korostelina (2005). This involved one dependent variable - readiness for conflict on behalf of one's ethnic group - and six measured independent variables: ethnicity (Crimean Tatar vs. Russian), salience of ethnic identity, salience of national (Ukranian) identity, belief that the national identity should be Ukranian, belief in a multicultural national identity, and belief that national identity consists of civic obligations. The investigator found that salient ethnic identity predicted high readiness for conflict and salient national identity predicted low readiness for conflict. There were also some complex interactions among her independent variables; for example, she found that, ‘possession of a salient national identity and of the ethnic concept of national identity strengthens the influence of salient ethnic identity on conflict readiness among Russians and weakens it among Crimean Tatars' (p. 103).
In theory, Korostelina could have manipulated her five psychological variables (the salience and belief variables) in a clever experiment.2 But in practice, this would have been difficult, as it would have required running participants in at least 32 experimental conditions (a 2 ? 2 ? 2 ? 2 ? 2 factorial design).3
A third advantage of correlational designs is that they allow us to examine the impact of one independent variable on another, a result that is not possible if these variables are manipulated. For example, Korostelina found that ethnic identity was stronger in Crimean Tatars than that in Russians.
A fourth advantage of correlational studies is that they can be used to evaluate the strength of the relationship between an independent and a dependent variable (Carnevale and De Dreu, 2005). This is not possible when the independent variable is manipulated, because good experimental design requires augmenting the strength and potency of the manipulated variables and reducing variation in the measured variables. This allows us to reach statistical significance if our variables are related to each other, but unnaturally magnifies the strength of that relationship. Experiments are for finding out whether variables are related to each other, not for determining the extent of that relationship.