Evaluation Review

Evaluation takes place in a social context that influences research design, selection of variables, the written report, and the timing of its release. There are also consequences for program implementors, for those subject to the program, and for the evaluators. Evaluations and evaluators may become involved in political conflict within the subject system and conflict external to it as well. The present study makes use of archival data to illustrate the issues in evaluations of the Gary plan of education that took place between 1914 and 1918. Suggestions for confronting political and social realities surrounding evaluation emerge from an application of concepts deriving from the sociology of knowledge.

The impact on gun-related crime of legislative changes involving Massachusetts' Gun Control Law is evaluated. Viewing crime rates as realizations from some underlying stochastic processes, a procedure for measuring intervention effects on crime rates is presented. Time series on gun-related crimes for the city of Boston are analyzed with this procedure in evaluating the impact of the gun control law.

Telephones were introduced as an innovation into a detention facility to help pretrial defendants raise bail money and thereby obtain release. Using an experimental design to evaluate the innovation, the results showed that persons who used the telephones were more likely to be released on bail than were those denied its use. Although the evaluation was successful, the implementation was not. The district attorney and court blocked its acceptance until a few years later when detained prisoners rioted and telephones were installed in order to reduce overcrowding.

This paperpresents a detailed evaluation of an experimental hospital psychiatryprogram: Individualized Patient Programming. Using concepts common to systems theory, the program emphasized the use of evaluation and feedback for increased program effectiveness. In order to make evaluation and feedback possible, and also to provide greater assurance of patients' receiving the best possible treatment, patients were assigned individually to case coordinators who were responsible for evaluation of the treatment program. Informationflow was facilitated through development ofa computerized clinical record and evaluation of patients' community adjustment and satisfaction with services. Results indicated partial success for the program with greater community adjustment compared with a no-treatment control program, and lowerrehospitalization ratesfor male patients. The successes realized through adoption of the experimental program were considered the result of increased individualization and evaluation of patients' programs and not necessarily considered to be realized through attempts at systematic feedback on outcomes .

A program training the elderly as home health aides is evaluated using a variety ofprocedures on a number of levels. Primary outcome variables center on graduate employment and graduate and patient satisfactions. Quality of instruction, interorganization cooperation, and legal issues also are examined. Questionnaires sent to graduate aides, interviews and observations with patients and caretakers, interviews with key informants, and the use of documents were all important data sources. Though the design was necessarily post hoc, we were able to make a number of useful observations about program successes and problems.

Three hundred students in grades 7, 8, and 9 wrote two achievement tests and one attitudinal test before the introduction of the "Responsibility" materials to the experimental group. After one week of teaching the unit on "Responsibility," the tests were again administered to the same students. A three-way (grades x ability groups x programs) multivariate analysis of variance of gain scores was performed to test the various hy potheses. Only the differences between programs were found to be significant; those differences could be attributed to a slight but significant increase in the scores of the experimental group students on the attitudinal test.

The present study was carried out to define the power and efficiency of thiocyanate as a discriminator for smoking status, and to apply some principles of decision and utility analysis to aid in the utilization of the diagnostic test in an intervention program. Data from 652 participants in the Multiple Risk Factor Intervention Trial (MRFIT) were used to derive the sensitivity and specificity of the thiocyanate test. Predictive values were computed based upon new prevalence estimates in order to determine the predictive validity of the test in various samples of smokers. The second part of the paper investigated the impact of various costs of errors of diagnosis upon an intervention program using thiocyanate as a discriminator variable. Implications for the use in intervention programs of thiocyanate as opposed to screening procedures are discussed.

Although interest in evaluation research is burgeoning, there is agreement about neither the character of evaluative activities nor the technical methods used in their application. The intent of this paper is to provide a broad taxonomy of assessment activities and methodological alternatives. Based on the first-order characteristic of evaluation pur pose, six assessment categories are identifted. It is argued that the choice of method should be contingent on the objective of the evaluation, and that a typology can serve as a basis for communication and assist in the choice of an appropriate methodology.

An alcohol education program and procedures for its evaluation are described. Various alcohol programs were administered to eighth grade students in ways to reduce Hawthorne effects and other experimental biases. Procedures, which were independently administered from the program delivery, are discussed whereby data were obtained pertaining to process and outcome variables. Obtaining other types of data as well is suggested in order to provide as much information as possible to assist in the interpre tation of results. Finally, various data analyses as well as sample size are considered.