Abstract
Linked article:
1 Background for the Review
Drug courts are based on an emerging theory of therapeutic jurisprudence that hypothesizes that therapeutic effects can result from judicial actions (Hora, 2002). In its simplest form, a drug court uses the power and authority of a judge to keep a drug offender in treatment, providing rewards for successes and sanctions for failures (U. S. General Accounting Office, 1997; Drug Court Professionals, 1997). Typically, a judge closely monitors the progress of a drug offender (generally referred to as a client) and doles out sanctions for drug use relapse, failure to attend treatment, or other drug court infractions. The judge also reinforces successes through praise and encouragement, and possibly a reduction in formal requirements or other “reward.” Depending on the structure of the drug court, successful completion may be accompanied with dropping the charges that brought the offender before the court (pre-plea/diversionary court) or expunging the offense from the record (post-plea court). Many drug courts also have a formal graduation ceremony for those successfully completing the program. The atmosphere of the drug court is non-adversarial and provides a case management function, connecting drug abusers with appropriate treatment programs. As described by the Drug Court Professionals (1997),
The judge is the central figure in a team effort that focuses on sobriety and accountability as the primary goals. Because the judge takes on the role of trying to keep participants engaged in treatment, providers can effectively focus on developing a therapeutic relationship with the participant. In turn, treatment providers keep the court informed of each participants progress so that rewards and sanctions can be provided. (Drug Court Professionals, 1997, p. 7)
The essential features of a drug court are the (a) integration of alcohol and other drug treatment and justice system case processing, (b) a non-adversarial courtroom approach, (c) random urine drug screens or other monitoring of abstinence, (d) judicial monitoring of a participants progress via status hearings, and (e) a system of sanctions and rewards for program infractions and achievements (Drug Court Professionals, 1997; Marlowe, Festinger, Lee, Dugosh, & Benasutti, 2006).
Drug courts are presumed to affect an offender's drug use and criminal behavior through both the actions and influences of the court and the involvement of the offender in mandated drug and alcohol abuse treatment (e.g., Marlowe et al., 2006; Banks, 2001). Drug involved offenders processed in a traditional manner by the criminal justice system are also routinely referred to drug treatment. However, treatment compliance is a major problem with this population and a clear impediment to successful outcomes (Simpson et al., 1997; Drug Court Professionals, 1997). The design and structure of drug courts is intended to address this problem by using the power of the judge to compel treatment compliance.
Drug courts began in Dade County Florida in 1989 as a diversion program for drug offenders. Since that time, drug courts have become popular throughout the United States. There were nearly 800 established or recently implemented drug courts in the United States as of January 2002, with many more being planned (Drug Court Clearinghouse, 2002).
The rapid expansion of drug courts throughout the United States criminal justice system has proceeded despite a lack of solid empirical evidence establishing the effectiveness of drug courts in achieving their aims of reduced drug use and ancillary criminal behavior. A review of the drug court literature conducted by the U. S. General Accounting Office (1997) concluded that the existing evidence was insufficient to draw any firm conclusion on the effectiveness of these programs with respect to recidivism.
More specifically, the U. S. General Accounting Office (1997) identified several limitations of the 20 evaluations examined, including a failure to examine outcomes beyond program participation and a failure to use a comparison group design. Twelve of these evaluations included a comparison group and six of these examined recidivism post-program. Summarizing these studies, the GAO stated that:
Some studies showed positive effects of the drug court programs during the period offenders participated in them, while others showed no effects, or effects that were mixed, and difficult to interpret. Similarly, some studies showed positive effects for offenders after completing the programs, while others showed no effects, or small and insignificant effects. (U. S. General Accounting Office, 1997, p. 85)
Belenko (2001) drew a cautious but positive conclusion on the impact of drug courts on long-term drug use and criminal offending based on a review of 37 evaluations. Not all of the evaluations reviewed by Belenko examined drug use or other criminal activity outcomes. Belenko was critical of the field's dearth of evaluations that examined post-program drug use and other criminal behavior, noting that only six of the studies he reviewed examined the long-term effects of these programs. The process data reviewed by Belenko suggested that “drug courts have achieved considerable local support and have provided intensive, long-term treatment services to offenders with long histories of drug use and criminal justice contacts, previous treatment failures, and high rates of health and social problems” (Belenko, 2001, p. 1).
A more recent systematic review of drug court evaluations was recently completed by the GAO (U. S. General Accountability Office, 2005). It examined 27 evaluations and concluded that “adult drug court programs led to recidivism reductions during periods of time that generally corresponded to the length of the drug court” (p. 5). The evidence on the effectiveness of drug courts to reduce substance use, based on this review, was mixed. This study also reviewed four cost-benefit evaluations and concluded that drug courts do yield a net benefit.
The rapid adoption of drug courts throughout the United States during the past 16 years has been phenomenal and reflects widespread belief in the effectiveness of drug courts at reducing criminal behavior. Prior reviews, however, draw cautions conclusions regarding the effectiveness of these programs. Numerous new evaluations have been completed in recent years, warranting a fresh look at the evidence.
2 Objectives of the Review
The objective of this review is to systematically review all existing evaluations of the effectiveness of drug courts with respect to future criminal offending and drug use behavior. At issue is the effectiveness of these programs relative to existing criminal justice system alternatives. The review will also critically assess the methodological soundness of the existing evidence and examine the relationship between features of the drug court and effectiveness (i.e., are some types of drug courts more or less effective than other types?).
3 Methods
3.1 Criteria for inclusion and exclusion of studies in the review
The population of studies eligible for this review is experimental and quasi-experimental evaluations of drug courts that utilized a comparison group. The criteria for inclusion are (a) that the study evaluated a drug court program (defined as specialized court for handling drug cases that is nonadversarial with a mechanism for referring offenders to appropriate treatment programs and a judge who actively monitors progress and provides sanctions for misbehavior); (b) that the study included a comparison group that was treated in a traditional fashion by the court system (e.g., probation with or without referral to treatment); and (c) that the study reported a measure of criminal behavior, such as arrest or conviction for some measurement period following the start of the program (the measure may have been based on official records or self-report and may have been reported on a dichotomous or continuous scale). These criteria are inclusive with respect to methodological quality, allowing for the inclusion of studies with clear methodological weaknesses. However, studies without a comparison group or that use drug-court drop-outs as the comparison group will be excluded. The quality of the methods will be coded and the robustness of the findings examined in relationship to this methodological variability. The form for establishing study eligibility is provided in Appendix A.
3.2 Search strategy for identification of relevant studies
The search strategy is designed to locate all eligible evaluations of drug courts, published or otherwise. Toward this aim, we will perform keyword searches of the following databases: Criminal Justice Periodical Index, Computerized Index to National Criminological Holdings (CINCH), Dissertation Abstracts Online, Government Publications Office Monthly Catalog, Government Publications Reference File, NCJRS, PsychINFO, Sociological Abstracts, Social SciSearch, and U.S. Political Science Documents. The keyword “drug court” will be used. This will cast a broad net that will be examined to identify potentially eligible studies. To identify unpublished evaluations, recent years (1999–2004) of the American Society of Criminology conference program will be reviewed for relevant studies. We also will contact researchers working in this field, including individuals at NPR Research, RTI, Glacier Consulting, Urban Institute, Center on Drug and Alcohol Research at the University of Kentucky. Finally, we will examine the bibliographies of literature reviews of the drug court literature (e.g., Belenko, 2001; U. S. General Accounting Office, 1997; U. S. General Accountability Office, 2005; Latimer, Morton-Bourgon, & Chrtien, 2006; Shaffer, 2006, and any other review articles we locate in the search process) and the bibliography maintained by the Drug Court Clearinghouse (http: //spa.american.edu/justice/drugcourts.php). The latter bibliography is updated regularly, is fairly comprehensive, and includes unpublished works. An initial search using these procedures has already been completed and has identified 51 eligible documents representing 38 unique studies (see reference list). We anticipate identifying several additional studies that have been completed since the time of the last search in April 2005.
3.3 Description of methods used in the component studies
The methods used by the studies included in this review are variations on a treatment versus comparison group research design. Experimental studies will have randomly assigned drug involved offenders to either the drug court or a regular court. The essential features of a drug court are defined above. Quasi-experimental studies will rely on naturally occurring groups of offenders processed by a drug court, by a regular court, or diverted from prosecution. Common quasi-experimental design variations may include historical controls, adjacent jurisdictions, offenders eligible for the drug court but who chose not to participate or for whom the prosecutor does not divert to the drug court, and eligible offenders who did not participate due to limited space in the drug treatment program. There is likely to be variability across studies in the nature of the comparison group. We will code the nature of the comparison group and any treatment components they receive (see the coding protocol for specifics). The studies will also vary with respect to the degree to which they employ statistical controls (matching, covariate analysis, etc.) to reduce the threat of selection bias. In all cases the participant samples will be adults or juveniles charges with a criminal offense. The influence of these methodological variations on observed effects will be examined through moderator analyses.
All studies included in this review will have a measure of continued involvement in either crime or drug use. That is, studies must have some measure of criminal involvement such as arrest, conviction, or incarceration, following entry into the program, or a measure of drug use, including arrest or conviction for a drug offense. Most typically, these measures will be official indicators of arrest or conviction. We anticipate that a small number of studies may have self-report measures of these construct, such as self-report measures of drug use. A few studies may also report the results of drug testing, such as urinalysis.
3.4 Criteria for determination of independent findings
There are two potential sources of nonindependence of findings in this synthesis. The first is multiple indicators of recidivism and drug use reported at multiple time points, that is, multiple effect size generated from a single study. The methods for handling multiple effect sizes from a single study to ensure statistical independence are detailed below in the section on statistical procedures and conventions. In short, no analysis or compilation of effect sizes will use more than one effect size per independent study sample.
The second potential source of nonindependence is the same data being reported across multiple documents. We will use author names, court location, and study time frame to identify multiple publications of the same evaluation. When such multiple publications are identified, the most complete and detailed manuscript will be designated as the primary coding source. Additional manuscript will be consulted to flush-out coding if necessary.
3.5 Details of study coding categories
An elaborate coding protocol was developed for this synthesis that provides for a systematic method of extracting information regarding each study's research design, program and offender characteristics, nature of the outcome measure, and outcome data. The protocol allows for the coding of any number of effect sizes for each drug-court/comparison-group contrast reported in a study. The method of handling the statistical dependencies this produces is discussed below. Also note that the protocol defined the drug-court/comparison-group contrast as the primary unit-of-analysis. The coding protocol is provided in Appendix B. All studies will be double-coded by independent coders. Coding differences will be resolved by the lead author.
3.6 Statistical procedures and conventions
The effect of a drug court on recidivism, including general criminal behavior and drug use, will be encoded using the odds-ratio. The odds-ratio is well suited to dichotomous outcomes such as those commonly used in drug court evaluations (e.g., percentage of the drug court and comparison group sample with an arrest or conviction at follow-up). When the measure of drug use or criminal behavior is measured continuously, we will compute a standardized mean difference type effect size and transformed it into an odds-ratio (see Hasselblad & Hedges, 1995; Lipsey & Wilson, 2001).
All analyses will be performed on the logged odds-ratio using the inverse variance method (see Lipsey & Wilson, 2001). We will assume that the true program effects estimated by the studies vary both as a function of measured between study differences (moderators such as method of assignment and type of drug court) and unmeasured differences. As such, we will implement a random-effects model, or in the case of moderator analyses, a mixed-effects model (Lipsey & Wilson, 2001; Raudenbush, 1994). These analyses. will be performed using program code written for Stata that is available at http://mason.gmu.edu/∼dwilsonb/ma.html.
Many studies will report data on more than one indicator of recidivism and a few studies will report on the same indicator at multiple follow-up points. The multiple effect sizes coded from a single drug-court/comparison-group contrast are statistically dependent (see Gleser & Olkin, 1994, for a discussion). Several methods exist for addressing this complication of meta-analytic data (Lipsey & Wilson, 2001). The approach that will be adopted in this synthesis will be to select one effect size for each of the following outcome types: (1) any future criminal behavior, (2) drug use behavior, and (3) non-drug use criminal behavior. Preference will be given to indicators that do not include technical violations as a component of the measure. If an outcome measure is reported for multiple follow-ups, the longest follow-up that retained 90 percent of the baseline sample will be used. Finally, any multiple effect sizes remaining within a study after implementing the a priori selection criteria, these will be averaged and the average will be used as the effect size for the primary analyses.
Moderator analyses will be performed using analog-to-the ANOVA and meta-analytic regression methods (see Lipsey & Wilson, 2001). Both substantive and methodological quality measure will be examined, individually and collectively. The latter will be achieve by creating a multivariate model with method variables (e.g., random assignment to conditions, attrition, use of statistical controls, etc.) to assess for the affect of these variables collectively on important substantive moderators. That is, the meta-analytic regression analysis will examine whether the effect of a moderator variable remains after controling for method variation.
3.7 Treatment of qualitative research
At this time we have no plans to include qualitative research in this systematic review. We would welcome any collaborator with expertise in the area of qualitative research to contribute a review of qualitative evaluations to this synthesis.
4 Timeframe
The protocol was developed out of prior meta-analytic work by the authors on the effectiveness of drug courts. Results from this prior work has been presented at several professional conferences (Wilson, Mitchell, & MacKenzie, 2002, 2004; Wilson, 2005) and a recent journal article (Wilson, Mitchel, & MacKenzie, 2006). This protocol updates this work. As such, once the protocol is approved the proposed systematic review can be completed in a timely fashion. More specifically, we anticipate that the review could be completed by August of 2007.
5 Plans for Updating the Review
We plan to update this review every three years in accordance with Campbell Collaborative guidelines.
6 Acknowledgments
We would like to thank the Jerry Lee Foundation for partial support of this project.
7 Statement Concerning Conflict of Interest
We do not have any conflicts of interest regarding drug courts. None of the authors work as part of a drug court or have any financial interest in promoting drug courts as a criminal justice system policy.
Footnotes
A Eligibility Checklist
| First Author's Last Name | _______ |
| Document Identification Number | _______ |
| Coder's Initials | _______ |
| Date Eligibility Determined | _______ |
| Document Status | _______ |
| _______ Eligible | |
| _______ Not Eligible | |
| _______ Relevant Review |
| Yes | No | |
|---|---|---|
| __ | __ | The study evaluated a drug court defined as specialized court for handling drug cases that is nonadversarial with a mechanism for referring offenders to appropriate treatment programs and a judge who actively monitors progress and provides sanctions for misbehavior. To be a drug-court, all of these features must be present. |
| __ | __ | The study included a comparison group that received routine criminal justice system processing. The essential feature is that the comparison group did not participate in the specialized drug court. Study design may be experimental or quasi-experimental. One-group research designs are not eligible. |
| __ | __ | The study participants were exclusively under the supervision of the criminal or juvenile justice system. There is no age or gender restriction for this review. |
| __ | __ | The study reported a post-program measure of criminal behavior, such as arrest or conviction. The measure may be based on official records or self-report and may be reported on a dichotomous or continuous scale. |
| __ | __ | The study was conducted in or after 1989, the year drug courts first emerged. |
| __ | __ | The document is not a quantitative evaluation study (i.e., this document does not report any data regarding the effects of a drug court). |
| __ | __ | The document is a review article relevant to this project (i.e., may have references to additional eligible studies or background information useful for preparing written manuscripts for this synthesis). |
Notes:
B Coding Protocol
Treatment–Comparison Code Sheet
A study may report on multiple independent evaluations, such as independent treatment–comparison group contrasts, or may have a design that includes multiple interventions of interest contrasted with a single control group. Code separately each treatment–comparison contrast eligible for the review. Note that multiple treatment groups must have independent (non-overlapping) participants. A single comparison group may be contrasted with multiple treatment groups (in this case, code item 11 of the Study Level Code Sheet as “1”).
Sample Level Code Sheet
A separate coding sheet is provided for sample level data. This allows for the coding of subsamples within a treatment–comparison contrast, such as males and females. In this systematic review, no subsamples will be coded. A treatment-comparison contrast may only have one sample.
Dependent Variable Code Sheet
Code separately each dependent variable reported in the study. A dependent variable measured at multiple time points should be coded only once. For example, arrest measured at 6-, 12-, and 18-months is treated as a single dependent measure of the construct arrest. For noncriminal behavior dependent variables, code only the outcome label (item 61) and 64 and 65. Do not code effect sizes for noncriminal behavior dependent variables.
Effect Size Level Code Sheet
Code a separate effect size for each contrast between the treatment and comparison groups for each dependent variable.