RESEARCH EVALUATING CONFLICT RESOLUTION INITIATIVES

In 1995, Deutsch wrote “there is an appalling lack of research on the various aspects of training in the field of conflict resolution” (p. 128). The situation has been improving since then.

While research evaluating CRIs may have begun primarily with research con­ducted on conflict resolution training, in the last few years conflict resolution evaluation research has expanded to include the development of tools, method­ologies, and research conducted on a range of initiatives including interactive con­flict resolution workshops involving politically influential parties from both sides in international conflicts (see D’Estree, Fast, Weiss, and Jakobsen, 2001; Fisher, 1997; Kelman, 1995, 1998), interethnic encounter groups (see Abu-Nimer, 1999, 2004; Maoz, 2004, 2005), and peace-building activities (see Lederach, 1997), to name a few. In this section, we offer a brief overview of some of the method­ologies and instruments developed and research conducted over the past few years. We begin with an example of an instrument created by D’Estree, Fast, Weiss, and Jakobsen (2001) to assess the short-, medium-, and long-term impacts of interactive conflict resolution and other similar initiatives.

A Framework for Comparative Case Analysis of Interactive Conflict Resolution

D’Estree, Fast, Weiss, and Jakobsen (2001) created a framework, grounded in theory and practice, designed to be used as a tool for evaluating CRIs.

The framework has four categories, and each category contains a set of criteria for assessing CRIs. The first category, “Changes in Thinking,” includes criteria regarding various types of new knowledge that participants may gain from involvement in a CRI, such as the degree to which participants are able to problem solve, frame problems and issues productively, and communicate effec­tively. The second category, “Changes in Relations,” includes various indicators that the relationship between the parties in conflict has changed, such as the extent to which parties are better able to engage in empathetic behavior, vali­date and reconceptualize their identities, and build and maintain trust with the other side. The third category, “Foundations for Transfer,” includes criteria for assessing how well a CRI establishes a platform for transferring the learning to participants’ home communities once the CRI has ended. The criteria in this category include the extent to which participants have created artifacts (such as documents describing agreements, plans for future negotiations, joint state­ments, etc.) and put in place structures for implementing new ideas, and the extent to which the CRI has helped create new leadership. The “Foundations for Outcome or Implementation” category includes criteria that assess the extent to which the CRI contributed to medium- and long-term achievements that occur between the parties. Such criteria include the degree to which relation­ship networks have been created; reforms in political structures have occurred; new political input and processes have been created; and increased capacity for jointly facing future challenges can be demonstrated. It is important to note that the categories and accompanying criteria are interrelated, not mutually exclu­sive, and are not meant to be used in a linear fashion.

The framework also includes a matrix that differentiates between temporal phases of impact and societal levels of intervention. The temporal phases of impact are the promotion phase, in which a CRI attempts to promote or catalyze certain effects (assessed during the CRI); the application phase in which the effects of the CRI are attempted to be applied or implemented in the parties’ home environments (assessed in the short term after the CRI takes place); and the sustainability phase, in which the medium- and long-term effects of the CRI are assessed. The societal levels of intervention enable evaluators to distinguish between effects that occur at the individual (micro) level, societal (macro) level, and the community (meso) level, in which the transfer of effects from the indi­vidual to the societal level often takes place. D’Estree, Fast, Weiss, and Jakobsen (2001) suggest using a variety of unobtrusive methods to collect data along the dimensions of their proposed frameworks, including interviews, surveys, obser­vations, content analysis, and discourse analysis.

The Action Evaluation Research Initiative

Another methodology that has been developed to evaluate a wide range of CRIs is called action evaluation research (Ross, 2001; Rothman, 1997, 2005; Rothman and Friedman, 2005; Rothman and Land, 2004). Action evaluation research refers to a process of creating alignment and clarification about the goals of a CRI with a variety of stakeholders as a way of monitoring and assess­ing the successful implementation of a CRI (Rothman, 1997, 2005; Rothman and Friedman, 2005). The action evaluation process centers on three main sets of questions: (1) What long- and short-term outcome goals do various stakeholders have for this initiative? (2) Why do the stakeholders care about the goals? What motivations drive them? For trainers or developers of the initiative, what are the theories and assumptions that guide their practice? (3) How will the goals be most effectively met? In other words, what processes should be used to meet the stated goals?

These questions form the baseline, formative, and summative stages of the research.

As the CRI is implemented, the action evaluation process enters the forma­tive stage in which participants reflect on the action that has been taken so far, refine their goals as needed, and identify obstacles that need to be overcome in order to achieve the goals. The formative stage is an ongoing process of refine­ment and learning rather than a discrete, one-time process. The methods used at the formative stage include an on-line project log in which members can communicate with one another about important events, problems, and ideas; an on-line diary in which participants communicate directly with the action evaluator about ideas and concerns; critical incident stories in which partici­pants enter particularly positive or challenging events into a project database; and interviews conducted with participants. Once again, the action evaluator feeds back the data collected to the group members and works with them to continue to clarify the goals of the initiative, monitor progress toward the goals, and direct future work. At this stage, a progress report is created that compares the data collected so far with the baseline stage goals. The report addresses questions such as towards what goals has observable progress been made? What new goals have emerged over time? Where have problems and obstacles occurred? The action evaluator helps participants assess the obstacles and make changes to address them as needed.

The summative stage occurs as a CRI reaches its conclusion or another nat­ural point at which it makes sense to more formally evaluate the results of the CRI. At this stage, participants use the goals created at the baseline and forma­tive stages to establish criteria for retrospective assessment of the CRI.

We now look at several research studies conducted to evaluate a variety of CRIs in different types of environments.

Comprehensive Peer Mediation Evaluation Project

The Comprehensive Peer Mediation Evaluation Project (CPMEP), conducted by Jones and her colleagues, involved twenty-seven schools with a student popu­lation of about twenty-six thousand, a teacher population of approximately fifteen hundred, and a staff population of about seventeen hundred (Jones, 1997). They employed a three-by-three design: three levels of schools (elemen­tary, middle, and high school); each level of school was exposed to either peer mediation only (a “cadre program”), peer mediation plus (a “whole school pro­gram”), or no training at all (“control groups”). The training and research occurred over a two-year period.

The following draws on the report’s summary of general conclusions:

• Peer mediation programs yield significant benefit in developing con­structive social and conflict behavior in children at all educational levels. It is clear that exposure to peer mediation programs, whether cadre or whole school, has a significant and lasting impact on students’ conflict attitude and behavior. Students who are direct recipients of pro­gram training benefit the most; however, students without direct training also benefit. Exposure to peer mediation reduces personal conflict and increases the tendency to help others with conflict, increases prosocial values, decreases aggressiveness, and increases perspective taking and conflict competence. These effects are significant, cumulative, and sus­tained for long periods, especially for peer mediators. Students trained in mediation, at all educational levels, are able to enact and use the behavioral skills taught in training.

• Peer mediation programs significantly improve school climate.

• Peer mediation effectively handles peer disputes. When used, peer mediation is very effective at handling disputes. There is a very high rate of agreement at all educational levels on satisfaction by both the mediator and disputants.

• The results do not support the assumption that whole school programs are clearly superior to cadre programs. The latter have a strong effect on students’ conflict attitudes and behaviors, and whole school pro­grams have a strong impact in terms of school climate. Based on this evidence, schools that cannot afford a whole school approach may secure similar, or even superior, benefits with a cadre program that is well-implemented.

• Peer mediation programs are effective at all educational levels.

It is important to recognize that not only was this study well-designed from a research point of view but also the conflict resolution training was well- designed and systematic. The training for the peer mediation only and peer mediation plus (whole school program) conditions is outlined here.

For peer mediation only:

• Grades trained included fourth and fifth (elementary); sixth, seventh, and eighth (middle); and ninth, tenth, eleventh, and twelfth (high school).

• Eighteen to twenty-four students were trained per school in year one, and a second group was trained in year two.

• A minimum of fifteen hours of training was given in elementary schools and twenty to thirty hours in middle and high schools. Training was completed in four weeks or less and took place at the beginning of the fall semester in each year.

• Each school had a site leadership team (SLT) of four or five adults, with at least one teacher, one nonteaching staff member, and (where possi­ble) one administrator. The SLT was responsible for day-to-day imple­mentation and oversight of the peer mediation program.

• Each training organization gave SLTs one day of front-end program implementation training and contacted SLTs biweekly to monitor program implementation, offer program activities, and debrief cases.

For peer mediation plus or whole school programs, in addition to the peer mediation training and activities, each whole school program received (1) cur­riculum infusion training and (2) conflict skills training for staff.

Regarding curriculum infusion:

• Six to eight hours of training were given, using a standardized curriculum.

• A select group of teachers were trained (voluntarily, although any teacher could attend the training), who committed to using at least one period (forty-five minutes) per week of class time to teach from the conflict curriculum.

• One or two teachers per grade in elementary schools and two teachers per grade in middle and high schools delivered the conflict training in curriculum infusion classes for at least one full semester following training.

• Participating teachers received ongoing contact and support from the training organization.

As for conflict skills training:

• It totaled six to eight hours of conflict skills training.

• It was offered to all adult staff of the school (teachers, nonteaching staff, and administrators).

• Curriculum infusion teachers and SLT members were required to attend.

Negotiation Evaluation Survey

The Negotiation Evaluation Survey (NES) is a time-delayed, multisource feed­back approach to assessment and development of collaborative negotiation training and its effects on individuals and groups (Coleman and Lim, 2001). This approach uses a modified MACBE model (motivation, affect, cognition, behavior, environment) (Pruitt and Olczak, 1995) to assess change at the individual level in terms of conflict-related cognitions, attitudes toward the use of cooperative and competitive strategies, affect and behaviors, and at the group level in terms of conflict outcomes and work climate. In order to correct for self-report bias inherent in many evaluation tools, the NES was designed as a multisource feed­back (MSF) tool, often referred to as a 360-degree feedback instrument (Church and Bracken, 1997). The MSF process elicits perceptions of a target person’s behavior from a variety of sources (in this study from questionnaires filled out by self, a close friend, a supervisor, and a subordinate).

The NES was used to evaluate the effects of the Coleman/Raider Collabora­tive Negotiation model, which was used in the twenty-hour Basic Practicum in Conflict Resolution course at Teachers College, Columbia University (see Chap­ter Thirty-One). In addition to using the modified MACBE model as an orga­nizing construct for the survey, the authors identified the elements of the Coleman/Raider model and translated the elements into specific training objec­tives and then into measurable constructs that form the basis of the actual items used in the NES.

The study used a Solomon four-group experimental design with two treatment groups and two control groups. Both treatment groups received the training and a posttest survey, but only one took the pretest survey. Neither control group received the training and both took the posttest survey, but only one took the pretest. No significant effects were found in any of the four groups from taking the pretest, and no interactions between pretesting and training were found.

Training was found to have a significant effect on participants’ collaborative negotiation behaviors, thoughts, feelings, attitudes, negotiation outcomes, and work climates. For example, as compared to participants who did not receive the training, those who received the training were found to have:

• More cooperative and less competitive attitudes toward conflict

• More use of opening, uniting, and informing behaviors (as opposed to attacking and evading behaviors)

• More constructive conflict outcomes and work climate one month after the training

• Fewer attacking and evading behaviors

• Fewer negative emotions

Regarding the multisource feedback approach, the study found that:

• Friends tended to be more candid evaluators of targets’ negotiation behavior, particularly as compared to subordinates and self-reports.

• Subordinates tended to be kinder in their ratings than friends and super­visors, perhaps resulting from power imbalances in their relationship to the target and the nonanonymous nature of the instrument.

• Supervisors tended to give more flattering evaluations than participants themselves, particularly in categories that might reflect equally on the supervisor’s skills (such as on conflict outcomes and work group climate).

In an extension of this evaluation research study, Lim (2004) conducted a study in which she had the participants engage in a two-party negotiation sim­ulation three weeks after taking the posttest. Participants’ behavior and attitudes during the simulation were measured by blind raters (who coded tape-recorded verbal exchanges) as well as by participants’ self-reports regarding their own and their negotiating partner’s behaviors and attitudes during the negotiation simulation. She found that, compared to those who did not receive training, participants who received the training established a more cooperative climate in the simulated negotiation, did a better job probing for (as opposed to ignor­ing) the other party’s needs, demonstrated better active listening skills, and agreed to outcomes that better addressed both parties’ interests. Lim’s study also replicated the Coleman and Lim (2001) study and its findings were similar to the original study.

The Process Evaluation Approach

In extensive field research, Maoz (2004, 2005) used a process evaluation approach that assessed the extent to which CRIs promote relationships, behav­iors, and interactions that fulfill the standards of social justice, equality, and fair­ness that they strive to achieve within the larger conflict setting. We describe here two separate but related evaluation research projects. The first project examined forty-seven CRIs that brought together Jews and Palestinians living in Israel in encounters supported by the Abraham Fund for Jewish-Arab coexistence. The project assessed the level of symmetry between the Jewish and Palestinian participants as well as facilitators in all forty-seven CRIs. Assessed symmetry was based on direct observations of each CRI by the evaluation research team who recorded interactions and verbal exchanges using a detailed coding sheet and instructions booklet. Each CRI was assessed on a scale ranging from one (maximum dominance of one side, in this case Jewish) to nine (maximum dominance of the other side, in this case Palestinian). A rating of five reflected symmetrical participation of Jews and Palestinians.

In addition, the CRIs were classified into two categories: CRIs that empha­size coexistence and similarities between the sides versus CRIs that emphasize the conflict and power relations between the sides. (The CRIs in the latter cat­egory generally emphasize the conflict and power relations in order to raise awareness about existing asymmetrical power relations and to modify partici­pants’ construction of identity).

The process evaluation of the forty-seven CRIs found that:

• In the vast majority (89 percent) of CRIs both Jewish and Palestinian participants were equally active.

• With respect to participants, there was no significant difference in the degree of symmetry in initiatives that emphasized coexistence models versus those that emphasized the conflict and power relations.

• However, with regard to facilitators, only 45 percent of the CRIs had full or near symmetry between Jewish and Palestinian facilitators. In 45 per­cent of the CRIs, Jewish facilitators were rated as having medium to great dominance, while only a small percentage of Palestinian facilita­tors were rated as having medium to great dominance.

• With respect to symmetry between facilitators’ participation, there was a marked difference between CRIs that emphasized coexistence and simi­larities between the parties versus CRIs that emphasized the conflict and power relations. Approximately one-third of CRIs in the coexistence cat­egory were rated as symmetrical while approximately two-thirds of CRIs in the power relations category were rated as symmetrical.