Collaborative,Participatory,and Empowerment Evaluation

Differentiating Among the Stakeholder Involvement Approaches

A long list of colleagues has recommended that evaluation approaches to stakeholder involvement be differentiated (Miller & Campbell, 2006; Patton, 1997, 2005; Scriven, 1997, 2005; Sechrest, 1997; Stufflebeam, 1994), and many have helped to define and identify similarities and differences among these approaches (Fetterman, 2001; Fetterman, Deitz, & Gesundheit, 2010; Fetterman, Kaftarian, & Wandersman, 1996; Fetterman & Wandersman, 2005, 2007; Rodríguez-Campos & Rincones-Gómez, 2013; O’Sullivan, 2004; Shulha, 2010).

However, Cousins et al. ask, “Why is it important to have sharp distinctions among these approaches and to whose benefit?” (p.14) and conclude, “we find the investment in compartmentalizing genres of collaborative, participatory, and empowerment evaluation unwarranted and ultimately unproductive” (p. 15). Our view is that it is the nature of science and good practice to be precise, define terms, and explain differences among similar approaches in order to build on knowledge and improve practice. Differentiation of approaches helps evaluators select the most appropriate stakeholder involvement approach in the field. Hence, we advocate for distinguishing among approaches as follows:

Misleading Characterization

Cousins et al. present a hypothetical about an evaluation approach that hands off the evaluation to the community without the assistance of an evaluator (p. 14). They use a discussion with Donna Mertens about “turning control over to stakeholders” as their example. In this conversation, she concluded it would be “chaos” and that there needed to be “a partnership rather than a relinquishing of responsibility on the part of the evaluator” (p. 14). This is a misleading characterization of any major stakeholder involvement approach. Empowerment evaluations, for example, place program staff and community members in control of the evaluation. However, empowerment evaluators do not abdicate their responsibility. They work with communities as evaluation coaches and critical friends. This should be apparent from Fetterman, Kaftarian, and Wandersman’s (1996) earliest writings to case examples in Stanford University’s School of Medicine (Fetterman et al., 2010) and Hewlett-Packard’s US$15 Million Digital Village empowerment evaluation (Fetterman, 2013).

Confounding and Comingling Terms

The Cousins et al. article confounds and comingles terms when they use “collaborative evaluation” (books have been written about the approach) interchangeably with “collaborative inquiry” (p. 14). Their language is further confounded by shifting terminology. For example, their claim that participatory evaluation “remains somewhat of an enigma to us” (p. 14) is perplexing since those same authors were proponents of the approach (before changing their terms of reference). In a previous article, Cousins and Whitmore (1998, p. 7) noted, “Cousins and Earl (1992, 1995) outlined an approach they labeled participatory evaluation, which built on the conventional stakeholder model by advocating joint ownership and control of technical evaluation decision making, a more penetrating role for stakeholders, and restriction of participation to stakeholders most closely connected with the program.” In another article, Cousins and Earl (1995) explain, “We believe our form of participatory evaluation strikes an appropriate balance between the needs of technical rigor and responsiveness in evaluation. Our approach differs from other forms of participatory evaluation in that it does not explicitly hold the goals of emancipating oppressed groups, ameliorating social inequities, or redefining power relationships.” In addition, they have replaced the term participatory evaluation with the label collaborative evaluation and called collaborative evaluation practical participatory evaluation. This practice of confounding, comingling, and shifting terminology adds noise and confusion to the conversation.

This is not a purely academic exercise. Imprecise language has implications for practice and also for research on evaluation. The use of a common language, or at least translating terms and not confounding, comingling, or changing them, facilitates communication, dialogue, and understanding. While recognizing that a foolish consistency is the hobgoblin of small minds (Emerson, 1993), there is some merit to using terms consistently.

Using Collaborative Inquiry as the Umbrella Term

Similarly, privileging one label (or approach) over all three approaches, such as stating that collaborative inquiry or collaborative evaluation is the umbrella term for all approaches (e.g., p. 14) or earlier stating that participatory evaluation was the appropriate term for all approaches (Cousins & Earl, 1995), simply muddies the waters and neglects work to date.

A more appropriate global or umbrella term is “stakeholder involvement approaches” that encompass collaborative, participatory, and empowerment approaches (among others). (This differs from a similar approach, stakeholder-based evaluation; Bryk, 1983 and Mark & Shotland, 1985; in which the evaluator is in charge, widespread agreement about program goals is lacking, all legitimate groups are involved, and stakeholder participation is limited.).

Collaborative and empowerment evaluators have accepted the umbrella terminology of “stakeholder involvement approaches.” In addition, Cousins and Earl (1995) placed participatory evaluation within the context of stakeholder-based models in the past: “participatory evaluation is an extension of the traditional stakeholder-based model.”

Although the term collaborative inquiry may be a suitable overarching term for some stakeholder involvement approaches, it does not capture the underlying values driving empowerment evaluation, including capacity building, self-determination, and empowerment. Moreover, collaborative inquiry typically implies evaluator control of the evaluation and consultation with the stakeholder. Empowerment evaluation is the opposite. The stakeholder being in control is primary and collaboration is secondary. Stakeholder involvement represents a more generic and descriptive term for all three approaches. Building on the work of others, engaging in the dialogue, or at least acknowledging previous work in the discourse, has proved to provide more light than heat and greater conceptual clarity in the long run.

Cousins et al.’s (2013) resistance (p. 14) and “discomfort” (p. 13) with defining terms, compartmentalizing approaches, and clarifying approaches is not in the best interests of participatory evaluation, stakeholder involvement approaches, or evaluation in general. Our disagreements with Cousins et al. (2013), however, should not be used to divide and weaken strong bonds and relationships. They should be used to refine and improve our efforts. There is an overlap between collaborative, participatory, and empowerment approaches in practice. Synergistic strength is a function of interrelated and reinforcing characteristics and features. We invite colleagues to continue to engage in critical reflection and dialogue in an effort to strengthen the quality of our work. It is our hope that our dialogue will continue to produce more light than heat in the field.