Evaluation & the Health Professions

In the present article, it is argued that there is a benefit to applying techniques of exploratory data analysis (EDA) to program evaluation. To exemplify this, an evaluation of a rehabilitation program for people with rheumatoid arthritis ispresented. The perceived health status of patients receiving intensive rehabilitation services from a major rehabilitation institute was compared with that of patients receiving customary office-based care over an 18-month period. The data were analyzed in a conventional way (analysis of variance) and then by way of EDA techniques (graphic display of medians and boxplots). The conventional analysis suggested that all patients improved over time and that intensive rehabilitation services provided no particular benefit or harm. The exploratory analysis showed that the distribution of the outcome variable was patently nonnormal, thus casting doubt on the validity of the conventional analysis. The EDA furthershowed that the rehabilitation group lagged behind the comparison group fora year, with a precipitious improvementat the 18-month period. Thissuggests that a selection factor was operating (i.e., those in the rehabilitation group could have been sicker) or that the patients in the rehabilitation group were made more aware of their condition by the intensive health services they received The EDA provided an important insight.

Studies have demonstrated the efficacy of mutual-help interventions for bereaved women, but there have been no controlled intervention studies of this type for men. The objective of this study was to assess the efficacy of a mutualhelp group intervention for new widowers, using a randomized controlled design. A community sample of 113 recruited widowers bereaved less than 12 months were randomly allocated into treatment (n = 61) and waitinglist control groups (n = 52). Treatment consisted of nine weekly semistructured peer group sessions, which focused on the griefprocess, diet, new relationships, exercise, and life-style issues. Analysis of variance of three psychological measures (GeneralHealth Questionnaire, Beck Depression Inventory, State-Trait Anxiety Inventory) and three social measures (Social Adjustment Scale, Social Support Questionnaire for both availability and satisfaction) showed significant improvement over time for all subjects, but no significant differences between the two groups over the observation period (baseline to 8 months). Possible reasons for no treatment effect include: men may take longer than anticipated to work through their grief with a delay in any measurable effect; the more recovered men dropped out early; and focusing widowers' attention on their own and others'grief may hinder an early (and measurable) recovery from their grief

Valid assessment of professional competence has proven to be an elusive goal. Objective tests, direct observation ofperformance, overall ratings of competence, and simulations have been tried and found wanting in one way or another. Objective test items are criticized as being unrealistic and therefore invalid. Direct observation tends to be very unreliable and therefore invalid. Simulations and overall ratings of competence share both of these flaws to some extent. Basically, you can't win. This article outlines some of the many ways to lose and some ways to cut these losses. In doing so, it proposes a general framework for evaluating the validity of measures of competence, and it uses this framework to examine the strengths and weaknesses of three approaches to the assessment ofprofessional competence: direct observation, simulation, and objective testing.

The decade of the 1980s witnessed a revitalization of free-market interest in the use of incentives and voluntary participation to promote activities in a wide range of fields. Because of its history of decentralized control over physician residency training, the state of New Jersey found such an approach appealing when it sought to restructure its graduate medical education system. Two statewide task forces spent a year developing policy changes designed to produce voluntary changes in such areas as the size and growth of the state system. However, a 2-year follow-up survey of the directors of state residency programs revealed little perceivable change.

The opinion of some test experts is that negation (e.g., not, except) should be used sparingly or avoided in the stems of multiplechoice items. In these items, examinees are required to select the one alternative that is false from among those that are true. However, negation continues to be used and frequently accounts for 15-20% of the multiple-choice items within a given medical examination. A 90-item multiple-choice examination was administered in various formats to approximately 200 nursing students in each of two consecutive academic years. The findings of the present study provide evidence that the use of negation in stems should be limited and that the single-response, negatively worded (SRNW) item should often be converted to a multiple-response, positively worded (MRPW) item. This recommendation is based on the inherent cuing (ie., eliminating some alternatives by comparative deduction) within SRNW items and on an increase in the reliability and concurrent validity of MRPW test scores.

The oral examination is one of the traditional measures of student performance during clinical clerkships. Other studies have compared oral exams, written exams, and clinicalperformance, finding an unequal correlation among them and poor reproducibility of scores among examiners. This study of a third-year class on three required clerkships found a stronger correlation between oral exam performance and cumulative grade point average (GPA) than had previously been reported between oral exams and written or clinical grades and also found high reproducibility across clerkships, both overall and within class quartiles. These findings argue for wider use of the oral exam as an evaluation instrument on clinical clerkships.

A demandfor betterdata about pharmacists as working health professionals led the authors to study the work patterns of 1,146 pharmacists over a 30-year period from 1959 to 1989. Work history data were used to determine gender-specific participation rates as a function of age, years in the work force, and date. Percent full-time (as opposed to part-time) work as a gender-specific function of age was also determined. Participation rates for males and females, widely disparate in the 1960s, have converged and are not significantly different. Graduates of the 1980s, both male and female, are remaining with pharmacy as an occupation at a significantly higher rate than earlier graduates. Percent full-time rates for women pharmacists are significantly lower than those for male pharmacists, although women have registeredsignificant increases in percentfull-time work in each decade from the 1960s to the 1980s. An unexpectedfinding was the overall reduced participation by graduates of the years 1970 to 1979. The findings of the study are helpful in explaining recent controversies about the adequacy of the supply of pharmacists. The form of the data bases constructed for this study could be used as a model for the study of other health professionals.