Evaluation Review

The United Kingdom Employment Retention and Advancement (U.K. ERA) demonstration was the largest and most comprehensive social experiment ever conducted in the United Kingdom. It examined the extent to which a combination of postemployment advisory support and financial incentives could help lone parents on welfare to find sustained employment with prospects for advancement. ERA was experimentally tested across more than 50 public employment service offices and, within each office, individuals were randomly assigned to either a program (or treatment) group (eligible for ERA) or a control group (not eligible).

This article presents the results of a multilevel nonexperimental analysis that examines the variation in office-level impacts and attempts to understand what services provided in the offices tend to be associated with impacts.

The analysis suggests that impacts were greater in offices that emphasized in-work advancement, support while working and financial bonuses for sustained employment, and also in those offices that assigned more caseworkers to ERA participants. Offices that encouraged further education had smaller employment impacts.

Plausible results are obtained identifying those particular implementation features that tended to be linked to stronger impacts of ERA. The methodology employed also allows the identification of which services are associated with employment and welfare receipt of control families receiving benefits under the traditional New Deal for Lone Parent program.

An important question in the design of experiments is how to ensure that the findings from the experiment are generalizable to a larger population. This concern with generalizability is particularly important when treatment effects are heterogeneous and when selecting units into the experiment using random sampling is not possible—two conditions commonly met in large-scale educational experiments.

This article introduces a model-based balanced-sampling framework for improving generalizations, with a focus on developing methods that are robust to model misspecification. Additionally, the article provides a new method for sample selection within this framework: First units in an inference population are divided into relatively homogenous strata using cluster analysis, and then the sample is selected using distance rankings.

In order to demonstrate and evaluate the method, a reanalysis of a completed experiment is conducted. This example compares samples selected using the new method with the actual sample used in the experiment. Results indicate that even under high nonresponse, balance is better on most covariates and that fewer coverage errors result.

The article concludes with a discussion of additional benefits and limitations of the method.

Public health emergencies have a significant impact on the health of citizens, the local economy, and society as a whole. Currently, in many parts of China, there is a widespread lack of the knowledge and skills required for emergency preparedness and self-rescue. By carrying out targeted health education and intervention activities, and by popularizing the knowledge and skills in health emergencies, the abilities of citizens to respond to public health emergencies and protect themselves, reduce property damage, and eliminate unnecessary panic may be greatly improved.

The knowledge and skills of the public to respond to public health emergencies after an intervention in Sichuan province were surveyed. The conditions before and after the intervention were compared, and the effect of the intervention was evaluated.

Four counties in Sichuan province were selected for baseline survey by means of a multistage random sampling method. Among the districts and counties, one of each was selected for baseline survey as an intervention group and another of each were selected as controls. After 1 year of intervention, respondents were selected from the intervention group by means of the method used for the baseline survey for effect evaluation. Results: After statistical analysis, it was shown that, after the intervention, the total awareness rate of knowledge to respond to public health emergencies increased from 42.6% to 58.4% among men and from 42.9% to 55.8% among women, both significant differences (p < .001). It was also shown that the total awareness rates of the 18–35, 36–45, 46–60, and 60+ age-groups, respectively, increased from 44.5%, 43.7%, 42.2%, and 37.8% to 62.1%, 56.0%, 56.8%, and 51.9%, which were also significant differences (p < .001). Intervention, educational level, occupation, discussion with the families, and disaster experience were the main influential factors for the knowledge and skills of sudden public health events.

The study results show that, after the intervention in Sichuan province, the knowledge and skills of the public to respond to public health emergencies were significantly improved. It is also shown that health education interventions must be developed in accordance with geographical features, the educational level of the population, age characteristics, and other factors.