EVALUATING INTERVENTIONS

Conflict management is both a theoretical and an applied field. Many of the methods reviewed to this point in the chapter are suited to addressing theoretical issues or debates. They also consist of tools that can be used to evaluate interventions designed to settle or resolve conflicts.

A first challenge is to answer the question: What is being evaluated? An intervention can be a single, well-defined treatment such as an electronic mediation delivered at the time of a negotiation impasse. It can also consist of a family of procedures such as a peacekeeping mission. The former type of intervention is easier to evaluate; comparisons with a no-treatment control group facilitate causal inference. The latter presents the larger challenge of ascertaining the contributions of different parts of the package. Another dis­tinction is between case-specific and generic evaluations. Programs designed for imple­mentation in specific schools, communities, or regions are evaluated in the context of those institutions or organizations. Administrators and policy makers are concerned about local impacts: Does it work? Much of the peer­mediation literature consists of program­specific evaluations. A more theoretical approach to evaluation is taken by those investigators (and practitioners) interested in types of interventions.

A second challenge asks about effective­ness. This includes addressing the summative (impact) question: Did it work? It also includes the formative question: How did it work? Summative evaluations turn on the distinction between short- and long-term impacts. A reduction in violence or stabiliza­tion may last only as long as the peacekeeping forces remain in the country. School violence may erupt again sometime after the peer mediation programs have ended. This reversal calls attention to the need to monitor situations for some time after the intervention. They raise questions about whether to return or to alter the approach taken by the initial mission or program. Formative evaluations ask about the reasons for success or failure of interventions. Alternative explanations for outcomes often emerge. For example, an increased number of court cases investigating corrupt practices suggest that the rule of law has been restored. They may also suggest that the level of corruption remains high despite the efforts of the peace-building team. School peer-mediation programs may reduce violence because of the climate created by their implementation rather than the specific techniques used. Explaining the reason for observed impacts is important. It helps evaluators understand why impacts occur and to re-consider the way their interventions are delivered.

A third challenge refers to the dynamics of interventions. The popularity of a contingency approach to conflict analysis is based on the observation that situations and perceptions change. Intervention may have different effects at different stages of a resolution process. Negotiation research has shown that situational influences change from one stage of the talks to another (Druckman, 1993).

These challenges raise issues about the value of evaluations. They are practical con­cerns for the client or sponsoring organization. They are also research concerns for the evaluator who is keen on making a more lasting contribution to the field. The practical concerns include a clear definition of the intervention or program being evaluated, identification of the stakeholders and their interests, developing both summative and formative indicators, assembling a team with an appropriate division of labor, providing briefings and draft reports that are sensitive to the political environment in which the evaluation is conducted, and insuring that the recommendations are implemented. Clarity in defining the intervention is also a concern for researchers. But, in addition, sampling, con­trol groups (including a matched “treatment”- no treatment comparison), and monitoring effects over time for assessing change are important considerations.

A layer of complexity is added to eval­uations when interventions conducted at a micro-level (interactions among disputants) are evaluated for impact at a macro-level (societal changes). This problem is illustrated by evaluations of problem-solving work­shops, where assessments of attitude change at a micro-level are linked to assessments of pub­lic opinion at a macro-level (Rouhana, 2000). A question of interest is whether changes in workshops surface in larger changes in their respective populations.

Evaluations can be performed in a vari­ety of ways. Useful recommendations can be made from the application of less- rigorous approaches to research design. One of these approaches is referred to as action research. Like other forms of evaluation research, this approach seeks to provide recommendations (action plans) for organization or community change. Unlike evaluation research, it promotes close collaboration between investigators and orga- nization/community members in all phases of the research process. The participants in the research do not simply provide information to the researchers; they influence the direction taken by the project by designing the change, observing what happens following the change, reflecting on these processes and conse­quences, and planning further action (Kemmis and Wilkinson, 1998). The emphasis of these activities on process places action research in the realm of formative evaluation. The central role played by the consumers of the project and its products renders this approach different than other formative evaluations. Although there is considerable flexibility in the way an action project is implemented, most are guided by a sequence of tasks discussed by Bassey (1998), Robson (2002), and Druckman (2005).

Senehi’s (2002) idea of constructive story­telling resonates with the connection between social change and conflict intervention.

Other forms of applied research on conflict come under the rubric of research con­sulting. This refers to projects driven by problems defined by clients. The analytical consultants on many of these projects are asked to perform a variety of roles. For example, four roles were implemented on a recent consulting project. The problem consisted of development of a framework for organizing the various parts of complex inter-governmental negotiations. The role of advisor turned into that of bridge-builder (between the consulting/policy and academic communities) as the lead investigator sought input from academic specialists familiar with frameworks and with specific cases. The tasks involved theoretical knowledge and technical skills. The former was needed to identify key features of negotiation and to organize them into a framework that connected them in space and through time. The latter role involved reliability testing, refinement, devel­oping weighting procedures, and performing analyses. The product was transmitted to the client in the form of a report that emphasized applications of the framework to familiar cases of negotiation. Training in a variety of methodological approaches goes a long way toward producing a useful product. It is also useful for making contributions to theory in the field of conflict analysis. I turn next to a discussion of the value of multi-method research.