American Journal of Evaluation

Factors influencing evaluation use has been a primary concern for evaluators. However, little is known about the current conceptualizations of evaluation use including what counts as use, what efforts encourage use, and how to measure use. This article identifies enablers and constraints to evaluation use based on a scoping review of literature published since 2009 (n = 47). A fulsome examination to map factors influencing evaluation use identified in extant literature informs further study and captures its evolution over time. Five factors were identified that influence evaluation use: (1) resources; (2) stakeholder characteristics; (3) evaluation characteristics; (4) social and political environment; and (5) evaluators characteristics. Also examined is a synthesis of practical and theoretical implications as well as implications for future research. Importantly, our work builds upon two previous and impactful scoping reviews to provide a contemporary assessment of the factors influencing evaluation use and inform consequential evaluator practice.

In the current socio-political climate, there is an extra urgency to evaluate whether program impacts are distributed fairly across important student groups in education. Both experimental and quasi-experimental designs (QEDs) can contribute to answering this question. This work demonstrates that QEDs that compare outcomes across higher-level implementation units, such as schools, are especially well-suited to contributing evidence on differential program effects across student groups. Such designs, by differencing away site-level (macro) effects, on average produce estimates of the differential impact that are closer to experimental benchmark results than are estimates of average impact based on the same design. This work argues for the importance of routine evaluation of moderated impacts, describes the differencing procedure, and empirically tests the methodology with seven impact evaluations in education. The hope is to encourage broader use of this design type to more-efficiently develop the evidence base for differential program effects, particularly for underserved students.

In the last three decades, the promotion of stakeholder involvement in evaluation has been gaining relevance in the Latin American and internationally, across varied agencies, institutions, and civic organizations. The 2030 Agenda and the Global Evaluation Agenda have also recognized the centrality of participation in evaluation. This article explores stakeholder involvement in evaluation based on collaborative work with stakeholders from 15 evaluative experiences. It shows what characterizes participatory evaluation in the region today and the principles of this practice.

Based on interviews and case studies in nonprofit organizations, this article examines the perspectives of evaluation advocates undertaking evaluation in culturally diverse contexts where interpersonal barriers may exist. Participants provided insights into how to overcome issues by developing meaningful interpersonal relationships, finding shared goals, and providing encouragement. This article presents practical strategies that anyone promoting evaluation may find helpful to develop positive interpersonal relationships and equitable and inclusive working environments.

This article reports on ways in which United Nations human rights treaties can be used as a normative framework for evaluating program outcomes. In this article, we conceptualize a human rights-based approach to program evaluation and locate this approach within the broader evaluation literature. The article describes how a rights-based framework can be used as an aspirational set of indicators for program evaluations to promote activities that align with internationally agreed-upon human rights norms. We then describe a case study of the evaluation through which this method was developed, including its sampling design, methodology, and findings. The United Nations International Children’s Fund (UNICEF) inclusive education evaluation described highlighted the need for conceptual clarity around what inclusive education is, and the importance of contextualized innovation toward meeting the educational rights of children with disabilities. Human rights perspectives and evaluation designs can help create such clarity, but should also be used with care.

Bhutan's overarching development paradigm of Gross National Happiness (GNH) promotes a harmonious balance between material and non-material dimensions. But Bhutan's evaluation practice has not yet adopted the principles of GNH, preventing evaluation findings and recommendations from aligning with the priorities of GNH. This article makes the case that Culturally Responsive Evaluation (CRE) is a potential way that Bhutan can ensure evaluation is aligned with GNH. Authors analyze a recent, critical evaluation of the Direct Employment Scheme (DES), initiated by the Bhutanese government to increase youth employment. Authors developed a CRE assessment tool to understand whether and how the DES evaluation conformed to CRE. Findings of the review focused on five themes: (1) understanding context, (2) seeking advice, (3) forming an evaluation team, (4) methodology, and (5) understanding evaluation findings. Authors conclude with a discussion about how a CRE approach would have better ensured evaluation responsiveness to Bhutan's interests. This article contributes to discussions about criticisms that evaluation in the Global South does not serve in-country needs. It also holds the promise of providing evidence about how evaluation can support achieving Bhutan's goal of Gross National Happiness.

The notions of indigenous knowledge and cultural philosophies are becoming ubiquitous in many social inquiries, and evaluation is no exception. Nonetheless, the pursuit to embed relevant indigenous philosophies in contemporary evaluation has yet to succeed. In this article, we discuss indigenous relational philosophies, approaches, and practices as they relate to evaluation. Using qualitative research approaches, we interviewed 43 Indigenous development leaders and other local representatives in three local government areas in Ghana. Utilizing evidence synthesis approaches through a triangulation process, we conclude that indigenous knowledge and other cultural ethos were distinct in community-based development evaluation processes. There was an elusive intersection between indigenous and contemporary evaluation paradigms. Indigenous evaluation has principles such as community spirit, mutual trust, self-organization, relational patterns or networks, “ubuntu” ideals, consensus building, and collective action that can complement contemporary evaluation for the effective and efficient evaluation of community development programs and social policies. We identify key indigenous elements and other indigenous relational assessment patterns to aid in the design of an indigenously driven relational evaluation framework. The evaluative competencies embedded in indigenous philosophies are vast, thus, a call for future research is proposed.