Abstract
Linked article:
Linked article:
Background for the Review
Research indicates that a substantial proportion of incarcerated offenders are drug dependent. Peters and colleagues (Peters, Greenbaum, Edens, Carter, and Ortiz, 1993) for example, reported that 56% of a sample of Texas inmates were diagnosed as having a substance abuse or dependence disorder during the 30 days prior to their incarceration. Similarly, a survey of jail inmates in Ohio found that 51% were currently drug dependent (Lo and Stephens, 2000). In fact, it is estimated that about 40% of all Americans who clearly need drug treatment are under the supervision of the criminal justice system (Gerstein and Harwood, 1990:7).
Drug dependence also appears to be common among incarcerated offenders in countries outside of the U.S. While international research assessing drug dependence among incarcerated offenders is more limited, the existing evidence indicates that drug dependence is common among incarcerated offenders in many nations. For instance, Bennett (1998) found that 45% of a sample of arrestees in five English cities reported being drug dependent at one point in their lives, and 33% reported being currently drug dependent. Likewise, 31% of inmates incarcerated in Canadian federal prisons and 43% of inmates incarcerated in provincial prisons were found to be drug dependent (Pernanen, Cousineau, Brochu, and Sun, 2002).
In the absence of effective substance abuse treatment it is likely that a high proportion of these drug dependent offenders will persist in crime. In fact, statistics reported by the Bureau of Justice Statistics indicate that among probationers frequent drug abusers were 53% more likely to be re-arrested than non-drug abusers (Bureau of Justice Statistics 1995: 26). As such, the period of time when an offender is incarcerated represents a crucial opportunity to prevent crime by intervening in the cycle of drug abuse and crime. Several aspects of correctional facilities (i.e., prisons, jails) make incarceration-based substance abuse treatment attractive. Perhaps most importantly, these facilities have the capacity to mobilize considerable coercive force to encourage substance abusing offenders to engage in treatment, many of whom otherwise would not do so.
Incarceration-based drug treatment is diverse, encompassing a broad array of treatment programs including group and individual psychotherapy, 12-step programs, methadone maintenance and punitive interventions, such as boot camps for drug abusing offenders. For our purposes, the defining features of these programs are that they target substance abusers, intend to reduce substance abuse and other criminal behaviors, and these interventions are based in a correctional facility. A preliminary search of relevant research revealed approximately 45 studies. These evaluations predominantly focused on assessing the effectiveness of therapeutic communities (TCs) and group counseling programs (e.g., drug education, 12-step programs, such as AA/NA). A considerably smaller number of evaluations have considered the effects of boot camp or methadone maintenance programs on drug users’ behavior.
The individual components of TCs vary widely. In order to create an environment conducive to rehabilitation, residents in therapeutic communities are most commonly housed in a separate distinct treatment unit away from non-participating inmates. Residents are instrumentally involved in running the therapeutic community including leading treatment sessions, monitoring other residents for rule compliance, maintaining the treatment unit, and resolving disputes. Staff and residents of TCs tend to be confrontational with rule violators, but residents also are supportive of each other's struggles to maintain sobriety. The guiding philosophy of TCs is that drug use is symptomatic of more general personal disorders, thus the focus of the treatment is on the underlying disorders and not drug abuse, per se.
Counseling/drug education programs are somewhat harder to characterize. Generally these programs incorporate elements of group counseling programs (e.g., 12-step programs such as AA/NA), life skills training, cognitive skills training, drug education, and adult basic (academic) education. The commonality among these programs is their reliance on group based therapies, in which substance abuse and other common problems are discussed among group members in an effort to solve mutual issues.
Boot camps are modeled after military basic training. Inmates participate in rigorous exercise regimens, learn military drill and ceremony, wear uniforms, and take on challenge courses (timed obstacle courses). Boot camps are highly structured: from the moment residents wake in the morning until lights out they are constantly engaged in scheduled activities. Boots camps also involve considerable confrontation, but unlike most TC programs confrontations most often occur between correctional staff and inmates—with drill instructors disciplining any deviation from established codes of conduct. In theory, the harsh, rigorous nature of boot camp programs serve as a deterrent to future criminal conduct, and the content of these programs instill self-discipline within program participants, which also leads to reduced recidivism.
Methadone maintenance programs are very different than other types of incarceration-based drug treatment programs. These programs attempt to solve the problems associated with heroin dependency (e.g., disease transmission, criminal activity) by prescribing methadone, a synthetic opiate. Unlike heroin, methadone does not produce a euphoric high; instead, methadone supplies a controlled amount of opiates into the client's blood steam that reduces opiate cravings. Furthermore, methadone blocks the euphoric high produced by heroin use. Long term methadone treatments gradually reduce the amount of methadone administered to the client until the opiate dependence is relieved.
Each of the above types of drug interventions ostensibly has the potential to reduce drug use and other criminal behaviors. An existing systematic review of this body of literature, however, only found strong evidence supporting the effectiveness of TC programs (Pearson and Lipton, 1999). In particular, Pearson and Lipton (1999) systematically reviewed the research assessing the effectiveness of corrections-based drug abuse programs in reducing recidivism. Their systematic review conducted a comprehensive search for quasi-experimental and experimental evaluations of interventions carried out in correctional settings [i.e., “prison, jail, or a similar residential correctional facility” (p. 390)], conducted in any country, and published between 1968 and 1996. Their search revealed 30 studies meeting their eligibility criteria. Pearson and Lipton's synthesis of the findings from these 30 studies indicated that boot camp and group-counseling interventions were ineffective in reducing recidivism among drug abusers. On the other hand, TCs were effective in reducing recidivism. Specifically, these authors’ analyses found that six of the seven TC studies reviewed produced substantial reductions in recidivism; the overall mean weighted r effect size was 0.133 (p = 0.025) with positive effect sizes ranging from .13 to .28 (one effect size was negative). In contrast, the mean effect size was not statistically significant for either boot camp or group counseling programs. Additionally, Pearson and Lipton found too few studies evaluating other types of interventions to draw strong conclusions; however, these authors characterized the evidence assessing the effectiveness of methadone maintenance, drug education, cognitive behavioral, and 12-step programs as being promising.
In many regards, this proposed systematic review is an extension of the work of Pearson and Lipton. The proposed systematic review, like the work of Pearson and Lipton, systematically and comprehensively reviews the effects of incarceration-based drug interventions on post-treatment drug use and other types of criminal behaviors using meta-analytic procedures. The primary substantive difference between their work and the proposed systematic review is that this research project uses a more current time frame (1980 through 2004). This is an important difference as in the last several years numerous evaluations of incarceration-based drug treatment programs were conducted. Spurred in large part by the Violent Crime Control and Law Enforcement Act of 1994, which provided Federal funds to states to develop and implement residential substance abuse treatment programs for persons held in state correctional facilities, many states established and evaluated such programs. The inclusion of these studies and other recent evaluations from other nations will yield a systematic review that captures the latest information regarding the effectiveness of incarceration-based drug treatment programs.
Objectives of the Review
The objective of this review is to systematically synthesize the available evidence regarding the effectiveness of incarceration-based drug treatment interventions in reducing drug use and recidivism. More specifically, this systematic review will focus on addressing the following research questions: Are incarceration-based drug treatment programs effective in reducing recidivism and drug use? Approximately how effective are these programs (i.e., what's the magnitude of the effect)? Are there particular types of drug treatment programs that are especially effective or ineffective? What program characteristics differentiate effective programs from ineffective programs? These questions will be addressed using meta-analytic synthesis techniques.
Methods
Criteria for inclusion and exclusion of studies in the review
The scope of this review is experimental and quasi-experimental evaluations of incarceration-based drug treatment programs for juveniles and adults that utilize a comparison group. The preliminary eligibility criteria for the proposed systematic review are that: (1) the study evaluated an intervention which was administered in a correctional facility; (2) the intervention targeted primarily substance users; (3) the evaluation used an experimental or two-group quasi-experimental research design which included a no-treatment or minimal treatment control/comparison group; (4) the study reported an outcome measure relating to criminal behavior (this concept includes drug use); and (5) the intervention was conducted between 1980 and 2004, inclusive. The last criterion was included to increase generalizability to current drug-involved populations. Our operational definition of “correctional facilities” includes jails, prisons, half-way houses, and other residential correctional facilities. In regards to the second criterion, interventions could include some non-drug involved offenders; however, at least 75% of offenders in a program must be drug users. We will determine the proportion of program participants who are drug-involved from the primary authors’ program description and descriptive statistics.
The third criterion specifies that all included evaluations must have a comparison/control group. Based on our preliminary search and review of the literature, in the vast majority of evaluations the comparison groups did not receive any specialized drug abuse treatment. However, in some evaluations the comparison group was involved in a drug abuse intervention; such evaluations will be included as long as the comparison group received an intervention that was clearly hypothesized by the researchers as being less effective and less intensive than the intervention received by the treatment group. For instance, in Magura et al.'s (Magura, Rosenbaum, and Joseph, 1993) evaluation of a jail-based methadone maintenance program for heroin addicts the comparison group received detoxification whereas the treatment group received stable doses of methadone during their jail stay and were referred to a methadone maintenance program in the community upon release. This study will be included in this systematic review as the comparison group received an intervention clearly hypothesized to be less efficacious than the intervention provided to the treatment group.
Stated differently, we will exclude quasi-experiments that involved comparisons of two or more interventions that were roughly comparable or whose comparability in terms of effectiveness in reducing recidivism was in dispute (i.e., treatment-treatment comparisons or dose-response evaluations). For example, the comparison group in Swartz, Lurigio, and Slomka (1996) was constructed by dividing program participants into four groups based upon length of program participation. Evaluations utilizing such comparison groups will not be included in this systematic review. Furthermore, we will not include evaluations in which the comparison group was comprised predominantly or solely of dropouts from the intervention of interest. For instance, evaluations such as Field (1985, 1989) which used program drop-outs as the comparison group will not be included in this systematic review. It should be noted that our preliminary coding forms record the type of intervention received by the comparison group; thus we will be able to examine empirically whether treatment effectiveness varies by type of comparison group used (e.g., no treatment vs. minimal treatment).
Combining results from quasi-experimental and experimental evaluations is controversial (Lipsey and Wilson, 2001: 9). Experimental evaluations uncompromised by attrition have higher levels of causal validity than quasi-experimental evaluations. Unfortunately, few studies in this body of literature have utilized experimental designs. In fact, our preliminary search of studies reveals that less than 10% of incarceration-based drug treatment evaluations employed an experimental design. Thus, excluding quasi-experimental evaluations would eliminate approximately 90% of all evaluations of incarceration-based drug treatment programs. Rather than exclude quasi-experimental studies, we have decided to include such evaluations and compare the findings of quasi-experimental evaluations to those of experimental studies in order to determine whether methodological rigor affects study findings.
Search strategy for identification of relevant studies
The goal of the search strategy is to identify all studies, published or unpublished, meeting the above eligibility criteria. In order to achieve this objective, a multi-pronged search strategy will be utilized. The search will begin by conducting a computerized keyword search of bibliographic databases. We have already conducted a preliminary search of the following databases: PsychLit, MedLine, NCJRS, Criminal Justice Abstracts, Dissertation Abstracts, Sociological Abstracts, Social Science Citation Index, SocioFile, Conference Papers Index, and UnCover. The keywords used were: drug treatment, drug counseling, therapeutic community(ies), methadone maintenance, boot camps, offenders, inmates, incarcerated, evaluation, and research. These keywords were combined in different permutations. Future searches will expand both the number of databases and keywords. We will also utilize reference lists of existing reviews of the literature. Additionally, we will also search the Campbell Collaboration Social, Psychological, Educational and Criminological Trials Register (C2 Trials Register), CINAHL, EMBASE, as well as SIGLE databases. Finally, we will solicit the aid of non-U.S. researchers to assist in the search of international evaluations, especially evaluations reported in languages other than English. All evaluations appearing to be potentially eligible based on a review of the title and abstract of each evaluation will be retrieved and closely scrutinized by two reviewers to determine final eligibility status.
Description of methods used in the component studies
The eligibility criteria details the methods used in the component studies. The basic research design for eligible studies is a treatment and comparison group design with a post treatment outcome measure of interest, such as criminal offending or drug use. The studies will vary with respect to the method of constructing the comparison group. A small number of studies will rely on random assignment of participants to the drug treatment and comparison conditions. The remaining studies will be observational, relying on groups constructed by natural means. The common variations will be historical controls, adjacent jurisdictions, offenders eligible for treatment program who chose not to participate, and eligible offenders who did not participate due to limited space in the drug treatment program. The studies will also vary with respect to the degree to which they employ statistical controls (matching, covariate analysis, etc.) to reduce the threat of selection bias.
Included research designs must also measure post program criminal behavior, such as official arrests, self-reported criminal behavior and drug use, and the results of drug testing (e.g., via urine analysis).
Criteria for determination of independent studies
Several types of statistical dependencies are evident in evaluations of incarceration-based drug treatment programs. First, it is common for evaluators to measure and analyze more than one indicator of criminal behavior (e.g., re-arrest, reconviction, drug use) or analyze the same indicator of criminal behavior at multiple post-program follow-ups (e.g., 6 months, 12 months). Second, it is also common for authors to report findings from the same sample of research participants in multiple reports. The statistical methods detailed below require statistical independence of study findings. We will utilize several strategies to maintain the statistical independence of study findings.
First, all studies will be cross-checked against one another to ensure that multiple studies reporting on the same sample of research participants do not contribute multiple estimates of program effects to any analysis. Second, in studies that report multiple measures of criminal behavior, the most general indicator of criminal behavior based on the full sample (i.e., not affected by attrition) will be selected. Guidelines for determining the most general measure are as follows: measures of any crime are more general than measures of specific types of offenses, self-reported measures of offending are more general than official measures; measures of re-arrest are more general than measures of re-conviction or re-incarceration; and, measures of re-conviction are general than measures of re-incarceration. If a sufficient number of each type of recidivism measure is encountered we will analyze each type of recidivism measure separately. These guidelines may be revised as we become more acquainted with this body of research. Furthermore, when multiple effect sizes are available, we will make use of several additional approaches to handing these dependencies including: utilizing only post-program effects covering the longest time period, examining post-program effects covering a standard period of time (e.g., 12 months), and averaging multiple post-program effects into one effect. 1 Third, any multiple effect sizes remaining from the same treatment/comparison group contrast will be averaged, and this mean effect size will serve as the effect size for the primary analyses. Fourth, a secondary set of analyses will be conducted that will analyze distinct types of criminal offending measures (e.g., drug use, drug offenses, property offenses).
Details of study coding categories
Preliminary coding forms have been developed for this project. These coding forms are based on coding forms utilized in previous meta-analyses conducted by D. B. Wilson and colleagues. These coding forms are structured hierarchically, in order to explicitly recognize the nested nature of effect sizes within studies. Any number of effect sizes can be coded from each treatment/comparison contrast using these forms (copies of these forms are included in the Appendix A).
The coding forms capture key features of the nature of the treatment, research participants, research methodology, outcome measures, and direction and magnitude of observed effects. Interventions will be categorized based on the primary authors’ description of the program, that is, we will categorized the program as a therapeutic community, methadone maintenance, 12-step program/group therapy, and such (see coding protocol. Our preliminary search indicates that most often, the authors self-identify the intervention into one of the categories we have established (i.e., therapeutic communities, methadone maintenance, boot camp, group counseling). It should be noted that if additional types of interventions are encountered we will modify our coding forms to include these new types of interventions. Programs typically include multiple modes of intervention, such as group and individual therapy or methadone maintenance and a 12-step program. To accommodate this blending of different program elements, we will also code for the presence/absence of individual program elements.
Two independent coders will assess each study. Discrepancies between coders will be resolved by one of the principal investigators. The uncorrected coding forms from each coder will be compared to examine interrater reliability. Only items with acceptable reliability (e.g., 70% agreement) will be utilized in the data analyses.
Coders will be trained by one of the principal investigators. First, coders will receive detailed instructions describing each element of the coding forms. Second, coders will be given a few studies that have already been coded to re-code for practice. One of the principal investigators will review the practice codings with each new coder, discuss any discrepancies between codings, and answer any questions new coders may have. This process will be repeated, until new coders demonstrate mastery of the coding forms.
Statistical procedures and conventions
An effect size will be calculated for each treatment/comparison contrast. We will utilize the odds-ratio effect size for dichotomous outcomes as this type of effect size is the most appropriate for dichotomous outcome measures (Lipsey and Wilson, 2001). Indicators of criminal behavior based on a continuous scale will be coded using the standardized mean difference effect size. These effect sizes will be coded in manner such that positive effect sizes indicate the treatment group had a more favorable outcome than the comparison group (i.e., less recidivism or drug use). The odds-ratio effect size (ESor
) is defined as:
Our analyses of these effect sizes will utilize the statistical approach outlined by Lipsey and Wilson (2001) and Wang and Bushman (1999). In particular, we will utilize the inverse variance method and assume that the true treatment effects vary as a function of both measured (i.e., coded study features) and unmeasured differences between studies. In order to capture unmeasured differences between studies, a random effects component will be added to the fixed effects weights calculated for each effect, as follows:
Our analyses will utilize macro programs written by D. B. Wilson. 3 These macro programs calculate the random effects variance component discussed above and compute various statistics such as the overall mean effect and the homogeneity of effects statistic. Further, we will also employ these macro programs to determine which study features are associated with observed study effects via meta-analytic regression, assuming a mixed-effects model estimated via maximum likelihood (Raudenbush, 1994; Overton, 1998).
Publication selection potentially biases the findings from a meta-analysis. Our decision to search for and include unpublished studies helps mitigate this potential source of bias. In addition, we will assess the distribution of effects for selection bias using the funnel plot and through application of the trim-and-fill method (Duvall and Tweedie, 2000).
Treatment of missing data
There are several potential sources of missing data in a meta-analytic systematic review, including unobserved studies, unobserved outcomes (effect sizes), and a lack of sufficient descriptive information in a written report to code moderator variables such as the presence/absence of various treatment components or participant characteristics. The search for unpublished studies helps mitigate the affects of the first of these. Furthermore, we will exam the distribution of effect size for publication-selection bias using both the funnel plot and the Duvall and Tweedie (2000) trim-and-fill method. These methods not only help detect publication bias but also bias resulting from missing outcomes. Missing moderator data will generally be handled by treating “missing” as an explicit category in analyses. This meta-analysis will not have a sufficient number of studies to justify the use data imputation methods.
Treatment of qualitative data
We do not have any plans to include the findings of qualitative research into this systematic review. However, we are open to suggestions from and collaboration with researchers specializing in such techniques.
Timeframe
We have conducted a preliminary search for evaluations assessing the effectiveness of incarceration-based drug treatment programs. Given the sources we have utilized thus far we are confident that we currently possess the vast majority of evaluations conducted in North America. In the next phase of the research, we will broaden our search to include international evaluations of such programs. We anticipate that this search will be completed by December of 2005. Study coding will be conducted continuously as studies are identified. Analyses will be performed in January and February of 2006. We anticipate that a written report conforming to the standard established by the Campbell Collaboration guidelines will be finished by March of 2006.
Plans for Updating the Review
Our plan is to update this systematic review every three years.
Acknowledgements
The preliminary search and development of this protocol have been supported in part by the Jerry Lee Foundation.
Statement Concerning Conflict of Interest
None of the authors have any financial interests in any existing or planned incarceration-based drug treatment or any competing types of interventions for drug using offenders. We do not believe that any conflict of interests will arise during the course of this research, and we will strive to avoid any potential conflicts of interest.
Footnotes
Appendix A:
1
We will also examine variation of treatment effects over time, in order to gain a sense of how long treatment effects persist.
2
Note that we will use the inverse of the odds-ratio, as we are interested in obtaining values greater than 1 to reflect a lower probability of recidivism in the treatment group relative to the comparison group.
