PROTOCOL: The Effects of Non-Custodial Employment Programs on the Recidivism Rates of Ex-Offenders

Types of Studies

Experimental and quasi-experimental evaluative studies will be reviewed, although these two sets will be analyzed and presented separately. Eligible studies must have included one or more treatment groups and one or more comparison groups. If subjects were not randomly assigned to the treatment and comparison conditions, the subjects must either have been matched or the study report must present evidence of initial group equivalence on demographic variables and prior criminal record. The comparison group may have received either ‘treatment as usual’ or no treatment. Studies will not be reviewed if the comparison group is comprised of persons who did not meet the eligibility criteria for the treatment. Comparison subjects may have been drawn from waiting lists or ‘treatment as usual’ pools; if the treatment group was comprised of subjects who volunteered to receive the intervention, the comparison group must also have been comprised of volunteers.

Types of Participants

Both the treatment and comparison groups must have been comprised, at least in part, of ex-offenders (i.e., persons who have been arrested, convicted, or incarcerated in connection with a criminal charge before becoming a study subject). If either the treatment or comparison group included subjects who were not ex-offenders, the results must have been reported so that effect sizes may be coded for the ex-offenders alone. Only studies of adult (as defined by the jurisdiction within which a given study is conducted) offenders will be eligible for this review.

Types of Interventions

Some of the treatment must have been delivered in a non-custodial setting (i.e., not in a prison or jail). Studies of treatment delivered in a halfway house, group home, or similar facility will be eligible. The program may have been either residential or non-residential so long as similar residential requirements were placed on both the treatment and comparison subjects. The treatment program must have included a job placement component, or a job training component although other components, such as life-skills training, remedial education, or social service assistance, may have been included. In the case of multiple service delivery, the non-employment components will be coded as well.. This will permit subsequent exploration of the extent to which the delivery of multiple components may have an impact on the effectiveness of the employment component. Types of Outcome Measures

Some measure of criminal behavior subsequent to the beginning of the intervention must have been reported for the ex-offenders in both the treatment and comparison groups. The measure of criminal behavior may have been either official (i.e., arrest, conviction, technical violation) or self-reported and may be reported either dichotomously or on a continuous scale. Post-intervention employment status will also be coded from study reports that included this information. Date and Language

Study reports written in English and completed during or since 1970 will be eligible for the review. Depending on availability of additional funds, studies in other languages may also be subsequently retrieved and coded. If none of the study subjects received treatment after 1964, the study will not be eligible.

Procedures for Eligibility Assessment

Two coders, yet to be named, will assess the eligibility of the studies independently. These judgments will be based primarily on an examination of abstracts and secondarily on a review of full-text reports in instances where either or both coders conclude that the abstract provides insufficient information to determine the eligibility of the study. Each coder will create a table indicating, for all studies retrieved, whether the study was categorized as eligible or not. They will then compare categorizations. Should there be differences between the coders regarding study eligibility, one of the co-principal investigators (Visher or Winterfield) will make the final decision.

Studies eligible for this review will include a mix of controlled experimental designs and quasi-experimental designs relying on subject matching to mitigate initial group differences. Some quasi-experimental studies may also present evidence of initial equivalence of the study groups without use of a matching procedure. The proposed eligibility criteria require that treatment group subjects and comparison group subjects not differ in their willingness to participate in the treatment prior to their assignment to study conditions. Studies that compare treatment volunteers with non-volunteers will not be coded in this review. Some of the eligible experimental studies (e.g., Rossi, Berk, and Lenihan, 1980) used fully crossed designs with a single no-treatment comparison group and multiple treatment groups. Including such studies in the quantitative analysis will require careful attention to issues related to the statistical independence of the effect sizes (see next section).

Determining Independence Of Findings

In any analysis, no more than one effect size from any subject sample will be included (Lipsey and Wilson, 2001). Multiple effect sizes from the same study may be included in a single analysis so long as the subject samples do not overlap. For example, a fully crossed experimental study involving three treatment groups and a single no-treatment comparison group would yield at least three (statistically dependent) effect sizes. All three effect sizes would be coded, but no more than one would be included in any given analysis. If a single study yields multiple effect sizes relevant to a single analysis (e.g., three measures of recidivism from a single job-placement study), the choice of which effect size to use will be made based on the content of the underlying dependent measure. In those instances where a content or theory-driven choice cannot be made, a random selection procedure will be used to determine which effect size will be included.

Study Coding Categories

Studies meeting the eligibility criteria will be coded for a range of characteristics related to methodological and programmatic quality. These characteristics will include: (a) the criteria used to identify units of analysis; (b) acknowledged violation or subversion of random assignment procedures; (c) criteria for selection of comparison subjects, (d) loss of participants from the study with particular attention to differential attrition between treatment and comparison groups; (e) description of intervention content, modality, and setting; and (f) the findings of any process evaluation addressing the fidelity of the program implementation to the intervention model.

All citations identified as potentially eligible for this review will be entered into a bibliographic database, and the full text will be retrieved. –A reasonable effort will be made to retrieve the full text of all studies identified as potentially eligible. We will document any failure to retrieve a study identified as potentially eligible.

Two reviewers will independently code each eligible study. Another reviewer will resolve any coding differences that arise between the two primary reviewers. If important information is missing from a study report, an attempt will be made to contact the original researcher(s) to determine if they can supply that information. Data will be coded directly into a database.

Statistical Procedures

We will conduct a quantitative synthesis of the coded studies separately for RCTs and QEDs. Many studies will report recidivism and employment outcomes on a dichotomous scale. Consequently, effect sizes will be computed as odds ratios. For studies that report recidivism outcomes on a continuous scale, effect sizes will be encoded as a standardized mean difference and transformed into an equivalent odds ratio. Mean effect sizes will be computed across studies using inverse variance weights, and the homogeneity of the effect sizes will be assessed using a Q test (Lipsey and Wilson, 2001). The between-study variance in effect sizes will be analyzed under a mixed effects model (i.e., fixed slopes and a random-intercept) including measures of both program content and study methodology. The model will be estimated using full-information maximum likelihood, as implemented in a macro developed for this purpose by Wilson (2002).