Abstract
Brief Introduction
This short book (145 pages from cover to cover) provides a very brief overview of the practice of program evaluation followed by a series of case studies for reflection and analysis. This contribution to the evaluation instruction literature has potential given the limited number and nature of most evaluation case study presentations for this purpose. An exemplar of most instructional case study is as seen in Program Evaluation: Methods and Case Studies (Posavac & Carey, 2006) where the material in question is generally limited to no more than a page or so and is included to illustrate a specific concept or aspect of practice. Program Evaluation in Practice goes beyond that presentation by providing both an instructional framework for treating the case study information and sufficient information about the project and its evaluation to enable not only specific illustrations and demonstrations but also more general ones.
Within the context of a graduate course in program evaluation, Spaulding’s effort is intended as a supplement to a main text and while not expected to serve as a replacement for actual experience, it is designed to provide students with the opportunity to “vicariously experience an evaluation project from beginning to end” and encounter and exercise decision making from the perspective of an evaluation project. According to the author, the case studies are composites of actual evaluation projects and real programs and although basing “a few characters on composites of fellow evaluators” the individuals depicted and the activities and events described in the case studies are fictitious.
Content and Structure
The book is divided into two parts. Part 1 includes a single introductory chapter, “Foundations of Program Evaluation,” which provides some orientation to and basic tools for the practice of evaluation as well as the organization and particular elements of the remaining chapters. The chapters of Part 2 comprise the bulk of the book and include the case studies illustrating evaluations of various programs: “Improving Student Performance in Mathematics Through Inquiry Based Instruction,” “Evaluation of a Community Based Mentor Program,” “Teaching Candidates Integrating Technology Into Their Student Teaching Experience,” “Evaluation of a Professional Development Technology Project in a Low Performing School District,” “Expansion of a High School Science Program,” “Evaluation of Proven Practice for Reading Achievement,” “Project Plan for Evaluation of a Statewide After-School Initiative,” “Evaluation of a Training Program in Mathematics for Teachers,” and “Evaluators in Training: Issues of Confidentiality, and Evaluation of a School Improvement Grant to Increase Parent Involvement.”
Throughout the case studies, examples, exhibits, and illustrations of surveys, interview guides and focus group moderator’s guides as well as other materials that would be expected or developed as part of the respective evaluation project are presented. The case studies showcase approaches from a variety of methodological and theoretical perspectives: goal free, participatory, logic models, and ex–post facto, to name a few. Each case is organized similarly beginning with learning objectives and concluding with Key Concepts, Discussion Questions, Class Activities, and Suggested Readings. Most of the case studies are structured by the following main headings: Evaluators, The Program, The Evaluation Plan, Summary of Evaluation Activities, and Findings and Final Thoughts, although occasionally there are sections related to that particular case study. All of the case studies deal with school-based and/or teaching-related programs, but the focus of each chapter can be more broadly applicable to general practice. For example, one chapter deals with issues of confidentiality, whereas another explores the purpose and application of needs assessment. The case study format allows for a contextualized consideration of many of the topics of interest and concern to educators/trainers of evaluators in training.
Critique
The book is a quick and easy read and that may be potentially deceptive to a cursory review. Although the contents of each case study do not present an exhaustive discussion or elaboration, they are of sufficient detail to allow for a “fair sense” of practice and likely enable students to consider “real life” issues, decision making, and consequences. An evaluation-course instructor will likely find that the presented case studies will enable an experience-based focus on many basic topics of practice that should provide a good basis for serious and significant class discussions about important issues. The “experience of evaluation from beginning to end” will depend on the context in which the text is utilized, the talent of the instructor, and on whether students go through the book from beginning to end.
However, for all its potential usefulness the book is somewhat limited by the relatively narrow scope of activities/programs that serve as the basis for the included case studies. It would have been interesting to have some of the same fundamental concepts considered and addressed from the perspective of programs and evaluations not all of which have to do with education or similar fundamental concepts considered across multiple contexts. Of course, there will be those who argue that the concepts that are addressed by the text are generally applicable regardless of context, theory, or application.
Given the intended purpose (i.e., “vicarious experience”) of the text, the descriptions of the hypothetical programs and attendant evaluation plans are very brief and generally insufficient for more than a superficial examination of evaluation plan development. Of course, the level of simplicity and superficiality may be by design since the intended audience is not expected to have much evaluation knowledge, skill, or experience. Finally, although the text endeavors to provide a “taste” of evaluation practice and may serve as a vehicle for activities to develop or exercise skill, it is clear that the main course text and the instructor will need to ensure that students have access to critical foundational information beyond what is presented.
Still, the book has potential in both traditional college courses and community-based settings. For students intimidated by the idea of “applied research” or anything that sounds like research, this book can serve to ease the angst. Instructors looking for a way to encourage class discussion about foundational issues in evaluation will likewise find the book useful, and students are likely to find the easy readability and limited information to their liking. Outside of the college classroom, the book might also be helpful in presenting evaluation topics in overview fashion to lay audiences with minimal technical language or theoretical emphasis. In whatever way the book is used, it will be best as a starting-off point for novice and unsophisticated audiences to consider evaluative practice in general and particular related concepts in more depth.