PROTOCOL: Cognitive-Behavioral Programs for Juvenile and Adult Offenders: A Meta-Analysis of Controlled Intervention Studies

The Rationale and Effectiveness of CBT for Offenders

One of the notable characteristics of chronic offenders is distorted cognition—self-justificatory thinking, misinterpretation of social cues, deficient moral reasoning, schemas of dominance and entitlement, and the like (Beck 1999; Dodge 1993; Walters 1990; Walters & White 1989; Yochelson & Samenow 1976). Offenders with such distorted thinking may react to essentially benign situations as threatening, e.g., be predisposed to perceive comments others make about them as disrespectful or attacking. They may hold conceptualizations of themselves, others, and the world that justify antisocial behavior, e.g., “nobody can be trusted,” “everyone is against me,” or “society doesn't give me a chance.” Their behavior may be guided by dysfunctional assumptions and rules about how one should behave, e.g., “you have to punish people for messing with you or they won't respect you,” “you have to rebel against authority or they will break you.” And, they may have deficient cognitive skills for long-term planning, problem solving, and decision-making that contribute to maladaptive and rigid behavior.

Cognitive-behavioral treatments (CBT) for offenders are designed to correct these dysfunctional and criminogenic thinking patterns. They employ systematic training regimens aimed at “cognitive restructuring” such that offenders develop more adaptive patterns of reasoning and reacting in situations that trigger their criminal behavior. For instance, CBT may train offenders to monitor their patterns of automatic thoughts to situations in which they tend to react with violence. Various techniques are rehearsed for assessing the validity of those thoughts and substituting accurate interpretations for biased ones. Often role-play or practice in real situations is used to help consolidate new ways of coping with situations that tend to prompt criminal behavior. CBT may focus on anger management, assuming personal responsibility for behavior (e.g., challenging offenders’ tendency to excuse their behavior by blaming the victim, society, or other circumstances beyond their control), taking a moral and empathetic perspective on interpersonal behavior (e.g., victim impact awareness), problem solving, life skill development, setting goals, or any combination of these themes. A relapse prevention component is also often included, which teaches offenders strategies for avoiding or deescalating the precursors to offending behavior (e.g., high-risk situations, places, associates, or maladaptive coping responses).

Prototypical examples of CBT programs for offenders include the following:

The Reasoning and Rehabilitation program (Ross & Fabiano 1985) is organized around a series of exercises (e.g., Critical Thinking, Social Perspective-Taking) that focus on “modifying the impulsive, egocentric, illogical and rigid thinking of the offenders and teaching them to stop and think before acting, to consider the consequences of their behavior, to conceptualize alternative ways of responding to interpersonal problems and to consider the impact of their behavior on other people, particularly their victims” (Ross, Fabiano, & Ewles 1988, p. 31).

Several reviews and meta-analyses have found that structured, directive, skill-oriented programs are generally more effective in reducing the subsequent reoffense rates of offenders than less structured programs (e.g., Andrews et al. 1990; Lipsey & Wilson 1998; Lösel & Koferl 1989; Palmer 1994; Redondo, Sanchez-Meca, & Garrido 1999). Cognitive-behavioral interventions appear to be especially effective among such programs. A recent meta-analysis of group-oriented cognitive behavioral programs for offenders examined 20 studies of varying levels of methodological quality and concluded that CBT was effective for reducing criminal behavior among offenders (Wilson, Allen, & MacKenzie, in press). Nearly all of the studies showed positive effects (thought not necessarily statistically significant) and representative CBT programs were found to reduce recidivism by 20-30% compared to untreated control groups.

The Wilson et al. meta-analysis was restricted to programs delivered in groups, but this is a minor limitation since virtually all such programs for offenders are conducted in a group format. A more interesting aspect of this meta-analysis is the range of offenders represented in the studies it included– some used general samples of offenders, others treated only specialized types of offenders, e.g., sex offenders, drug offenders, DUI cases, or batterers. Much of the variability in effects found across studies may have been due to differences in the response of these different types of offenders, though there were too few studies in any one category for Wilson et al. to closely examine this factor.

The only other meta-analysis of the effectiveness of CBT programs on the reoffense rates of offenders of which we are aware is Pearson, Lipton, Cleland, & Lee (2002). This meta-analysis included 69 research studies but covered both behavioral (e.g., contingency contracting, token economy) and cognitive-behavioral programs. Pearson et al. found that cognitive-behavioral programs were more effective in reducing recidivism than the behavioral ones, with a mean recidivism reduction for treated groups of about 30%. Moreover, studies of higher methodological quality showed the largest effect sizes. The criteria for identifying cognitive-behavioral programs in this meta-analysis were rather broad, however. They included not only interventions directed specifically toward altering cognitions, but also social skills training and problem-solving programs for which cognitive change was not the main focus.

These two meta-analyses described above provide strong indications of the effectiveness of cognitive-behavioral treatment for reducing the recidivism of offenders. However, both encompass considerable diversity within their scope, including a range of offender types, quality of study design, and (especially in Pearson et al.) variations in what is counted as a cognitive-behavioral treatment. Against this background, it seems wise to confirm the effectiveness of cognitive-behavioral treatment for offenders with an analysis of the set of available studies that most directly and convincingly tests this promising program approach. With that objective in mind, Lipsey, Chapman, & Landenberger (2001) conducted a meta-analysis of 14 studies of the effectiveness of CBT that were selected to have the following characteristics: (1)

Experimental or strong quasi-experimental design so that the most methodologically credible evidence would be represented;

The overall results of this meta-analysis showed that the odds of recidivating for offenders receiving CBT were only about 55% of those for offenders in the control groups. Examination of selected moderator variables further revealed that demonstration programs set up by researchers for research and demonstration purposes produced larger effects than “real world” practice programs. In addition, the effects for offenders treated while on probation were larger than those for offenders treated while incarcerated. Practice programs implemented on a more or les routine basis in correctional facilities, therefore, showed the smallest effects on recidivism but those effects were, nonetheless, positive and statistically significant.

Criteria for Inclusion and Exclusion of Studies in the Review

Studies will be assessed and selected for inclusion in the meta-analysis based on the following criteria (detailed criteria are presented in Appendix A):

Publication. Eligible studies may be either published or unpublished.

Country of origin. Studies may be conducted in any country and be published in any language.

Intervention. The treatment under investigation must be directed toward changing participants’ distorted or dysfunctional cognitions or toward teaching new cognitive skills in areas where participants have deficits. The therapeutic techniques should involve specific, relatively structured learning experiences designed to affect such cognitive processes as interpreting social cues, reasoning about right and wrong behavior, and making decisions about appropriate behavior. When the program also includes other elements, the cognitive-behavioral component should be centrally featured rather than only a secondary component. (A detailed definition of CBT is provided in Appendix A.)

Participants. The recipients of the intervention must be criminal offenders, either juveniles or adult, who are treated while on probation, incarcerated/institutionalized, or during aftercare/parole. They should be drawn from a general offender population and not be selected for, or restricted to, those committing specific types of offenses (e.g., sex offenses, DUI) or problem behaviors (e.g., drug use).

Outcome measures. The study must report subsequent delinquent or criminal offending as an outcome variable. It should also present quantitative data or statistical information that permits computation or reasonable estimation of an effect size statistic representing the contrast between the subsequent criminal behavior of treated versus untreated offenders.

Research methods. The study must use a design in which participants are assigned to intervention and control conditions either randomly or with a nonrandom procedure that does not involve manifest differential selection. That is, if nonrandom assignment is used, the selection procedure must not involve treatment-related differentiation (e.g., volunteers vs. nonvolunteers, treatment completers vs. dropouts) and the groups must be matched explicitly or implicitly on key demographic variables and/or prior offense histories or evidence must be provided that indicates the degree of initial equivalence between the groups on such variables. Control groups can represent placebo, wait-list, no treatment, or “treatment as usual” conditions, with the latter restricted to cases of clearly routine probation, institutional, or aftercare/parole practices.

Search Strategy for Identification of Relevant Studies

Initial library. An initial set of 14 studies eligible under the above criteria was assembled and analyzed for the Lipsey, Chapman, and Landenberger (2001) paper. This set of studies will be expanded through a comprehensive search using the procedures described below.

Database searches. Computerized bibliography searches will be conducted, mainly through the DIALOG SYSTEM. The search will cover studies reported during the period from 1965 to the present date. To the best of our knowledge, the first report of the application of CBT to offenders was by Yochelson and Samenow (1976), who described work conducted over the 5-6 years prior to publication of their volume. We thus don't expect to find eligible studies prior to 1970, but will search back to 1965 to ensure none are missed.

The keywords that will be used for searching the computer bibliographies will involve concatenations of words that describe the population of interest (e.g., inmates, offenders, parolees), CBT treatment (e.g., cognitive, CBT, criminal thinking), and effectiveness research (e.g., outcomes, evaluation, effectiveness). The keywords for the initial search are identified in Appendix B, but others will be added as they are identified during the search process itself.

Electronic databases to be searched include:

Cochrane Controlled Trials Register

Campbell Collaboration. Campbell Collaboration members will serve as a network to help locate eligible studies. The Campbell Collaboration Social, Psychological, Educational and Criminological Trials Register (C2-SPECTR) will serve as an additional resource.

International contacts. Our efforts to find studies from outside the U.S. will include corresponding with researchers and criminal justice personnel in countries known for their efforts in offender rehabilitation and research (e.g., Australia, Canada, German speaking countries, New Zealand, Scandinavian countries, South Africa, Spain, UK, and others) and ask for their suggestions and help in locating eligible studies, particularly unpublished and fugitive literature. Recent reviews on offender rehabilitation approaches in countries outside the U.S. will provide additional leads and addresses of contact personnel (e.g., 2001 series of international reviews available via Sciencedirect.com).

Cross-referencing of bibliographies. The references in relevant review articles, meta-analyses, and primary studies will be scanned for new leads.

Conference programs. Recent conference programs will be searched for eligible research presentations and posters at professional meetings of such organizations as the American Evaluation Association, the American Psychological Association, the American Psychological Society, the American Society of Criminology, the Academy of Criminal Justice Sciences, the International Society of Criminology, the American Sociological Association, the Society for Prevention Research, and others. Candidate papers will be requested directly from the authors.

Internet searches. Relevant government websites (e.g., NIJ, NIC, OJJDP, Home Office), foundation, professional associations and policy research firm websites will be searched. In addition, keyword searches will be conducted using search engines such as yahoo.com or google.com.

Electronic journals. Vanderbilt University subscribes to a large number of electronic journals (e.g., via OVID and Sciencedirect.com). The keywords listed in Appendix B will be used to search the article database. Keyword searching the entire document may lead to eligible studies that were missed in abstract searches.

Preliminary screening. Once a candidate study is identified, a preliminary screening will be made on the basis of title, abstract, and other available information (e.g., authors known to be conducting intervention research). For those reports that appear promising, an attempt will then be made to examine a full copy for closer scrutiny.

Study retrieval. Available sources for reports are the Vanderbilt University libraries, interlibrary loan, authors and sponsoring agencies, purchases from University Microfilms, book and journal publishers, and on-line journals.

Appendix C provides a preliminary bibliography of potentially eligible studies. It includes studies identified in Lipsey, Chapman, and Landenberger (2001) plus other candidate studies located since.

Description of Methods Used in the Primary Research

Primary studies in the field of offender treatment vary widely with regard to implementation and methodological quality of their research design. The highest quality outcome studies employ individual-level random assignment designs, monitor implementation throughout the delivery of the program, and measure reoffending at multiple time points over a period of two years or longer. Nearly all studies report outcomes as the percentage of reoffending participants in the treatment and control group.

Based on the previous meta-analysis of CBT programs (Lipsey, Chapman, & Landenberger, 2001), it is expected that about half of eligible studies employ a random assignment design, about two thirds provide sample sizes up to 100 (treatment plus control), half use rearrest and the other half reconviction as the outcome measure, and about two thirds of studies report recidivism over the first year post-treatment with the remaining third following participants longer. Ninety percent of studies are conducted with primarily male offender samples, including either Anglo or mixed samples (half of studies do not report the ethnic mix). Adult offender studies are about as frequent as juvenile offender studies. Demonstration programs are about as frequent as ongoing practical programs. In about two thirds of studies, treatment is delivered in correctional settings with one third providing CBT to participants in the community (probationers or parolees). More than three quarters of studies examine CBT programs that are delivered in a group format for a duration of 5-20 weeks and a total of 5-80 treatment contact hours.

Criteria for Determination of Independent Findings

This synthesis will examine only outcomes related to subsequent delinquent/criminal behavior (recidivism). Some studies will report multiple measures of recidivism (e.g., arrest, conviction, probation violations, disciplinary infractions, or breakdowns of different types of offenses). When more than one such outcome is reported, only one will be selected for the analysis. To keep as much comparability as possible across the studies, coders will select the outcome measure that is most frequently represented in other studies in the collection. Based on the pool of previously reviewed studies, it is expected that rearrest and reconviction are the two most common measures and will be the coders’ first choice if available; if not, coders will select the next most frequently represented measure of criminal behavior available. In some studies, recidivism will be reported for different time intervals, e.g., 6-months and one year post treatment. In those cases also the measure with the timing closest to that most commonly used across all the studies will be chosen to maximize comparability between studies. Moderator analysis with method variables will be conducted to determine if there are systematic differences in the measured outcomes as a function of the way recidivism is measured or the timing of those measures.

Details of Study Coding Categories

Coded variables. Appendix D provides a list of variables that will be coded as part of this review. This list may be expanded if deemed necessary in the early stages of coding. Particular emphasis will be placed on coding those moderator variables that proved relevant in our previous review paper (Lipsey, Chapman, & Landenberger, 2001). These will be used to break out study subgroups to contrast and compare findings.

Outcomes excluded. This proposal looks exclusively at outcomes representing reoffending. Studies may report additional outcomes, but these are not within the scope of the proposed review and will not be coded. However, we will record the frequency of these measures and so they may be considered for inclusion in larger-scale meta-analytic projects.

Coding reliability. All studies will be coded by two coders (one of the authors and a trained research assistant). Reliability will be based on the consistency of coding in the entire sample of studies. If specific coding items are found to be particularly unreliable, they will be reviewed and recoded by one of the authors. Remaining discrepancies will be determined by consensus.

Statistical Procedures and Conventions

Virtually all of the eligible studies we have collected so far reported outcomes as simple proportions or percentages of offenders in each research condition who recidivate. This information can conveniently be coded in odds ratios, representing the odds of recidivating among the treatment group relative to the odds among the control group. The odds ratio provides an effect size statistic for meta-analysis that has favorable statistical properties and yields readily interpretable results (Lipsey & Wilson 2001). For study reports that do not report recidivism outcomes as proportions, we will estimate the odds ratio if possible using whatever statistical information is reported for the intervention vs. control difference; e.g., for means of continuous variables we will compute the standardized mean difference and convert it to an odds ratio using the Cox transform described in Sanchez-Meca et al. (2003).

The statistical analysis will be conducted using SPSS and a set of SPSS macros developed by David Wilson (Lipsey & Wilson, 2001). Random effects analysis models will be used with the random effects variance component estimated using method of moments or maximum likelihood procedures. Effect size outliers (> ± 3.0 standard deviations) will be Winsorized to less extreme values (next highest not judged an outlier). Exceptionally large sample sizes will also be Winsorized to generate the values used in calculating effect size weights so that the corresponding effect sizes will not exercise a greatly disproportionate influence on any weighted analysis. Small proportions of missing data on variables other than effect sizes will be imputed based on the mean for the most similar studies. When larger amounts are missing, the variable will not be used in the analysis. In all cases, attempts will first be made to contact the original researchers to determine if they can supply the missing information.

Confidence intervals around each study's odds ratio will reveal which are statistically significant. The random effects weighted mean odds ratio across the studies, and its confidence interval, will be computed. To examine the variation in effects across studies, a Q-test of the homogeneity of the logged odds ratios will be performed (Lipsey & Wilson, 2001; Shadish & Haddock, 1994). Because a relative small sample of studies is expected to be available, the Q test will not have great power for rejecting homogeneity. If the Q test emerges as even marginally significant (p<.10) under these circumstances, further investigation of effect size variability will be undertaken.

One potentially important moderator of effect sizes that will be analyzed is type of research design. Although only randomized and strong quasi-experimental designs will be eligible for inclusion in this meta-analysis, they may yield systematically different findings. If so, design type will be used as a control variable in the analysis or, if the differences are too great or distinctly nonlinear, randomized and quasi-experimental studies will be analyzed and reported separately.

Sensitivity analyses will be conducted to determine whether the handling of outliers, imputation of missing data, or other such analysis decisions have appreciable influence on the results. These will be conducted by running the relevant analyses multiple times under different assumptions and procedures on these matters and comparing the results.

Treatment of Qualitative Research

The review will not include exclusively qualitative studies. Some qualitative information presented in reports of eligible experimental and quasi-experimental studies (e.g., regarding problems of treatment implementation) will be included in the form of ratings made by the coder.