Drug Courts' Effects on Criminal Offending for Juveniles and Adults

3.1 CRITERIA FOR INCLUSION AND EXCLUSION OF STUDIES IN THE REVIEW

The population of evaluations eligible for this review was experimental and quasi-experimental evaluations of drug courts that utilized a comparison group. In brief, the criteria for inclusion were that: (1) the evaluation examined a drug court program (defined as specialized courts for handling drug-involved cases that are processed in a non-adversarial manner, refer offenders to appropriate treatment programs, regularly test offenders for drug use, and have a judge who actively monitors progress and provides sanctions for misbehavior); (2) the evaluation included a comparison group that was treated in a traditional fashion by the court system (e.g., probation with or without referral to treatment); (3) the evaluations reported a measure of criminal behavior, such as arrest or conviction for some measurement period following the start of the program (the measure may have been based on official records or self-report and may have been reported on a dichotomous or continuous scale); and, (4) enough information was reported to compute an effect size. The form for establishing study eligibility is provided in Appendix A.

In regards to the first eligibility criterion, our operational definition of a “drug court” included specialized courts or court dockets that fit the five essential features of drug courts listed above (see pages 6-7). The task of identifying drug courts was made easy by the fact that nearly all of the interventions meeting this criterion self-identified as an evaluation of a “drug court.” The only ambiguity regarding this criterion concerned speedy case processing drug courts (e.g., see Belenko, Fagan, Dumanovsky, and Davis, 1993) and evaluations of the Breaking the Cycle (BTC) demonstration project (e.g., see Harrell, Mitchell, Hirst, Marlowe, and Merril, 2002). Speedy case processing drug courts were ruled ineligible because they focused on expedited case processing of drug cases, not substance abuse treatment with judicial monitoring of drug-involved cases. Evaluations of the BTC demonstration project were also ruled ineligible. While this intervention was based on the drug court model, a judge generally did not actively monitor clients and most clients, rarely, if ever had a status hearing. ⁴ Perhaps the clearest indication that BTC demonstration project was not a drug court program is the fact that a recent multi-site evaluation of drug courts used Pierce County's BTC program as a non-drug court, comparison program (see Rossman, Rempel, Roman, Zweig, Lindquist, Green, Downey, Bhati, and Farole, 2011).

The second criterion specified that all included evaluations must have a comparison/control group that received standard criminal justice processing. Essentially, this criterion required all evaluations to have a comparison and the comparison group received no drug treatment or minimal drug treatment. We excluded evaluations that compared drug court treatment to another drug treatment program of similar intensity (i.e., treatment-treatment comparisons or dose-response evaluations). Furthermore, we excluded evaluations in which the comparison group was comprised predominantly or solely of dropouts from the drug court. Evaluations that utilized program drop-outs as the comparison group were excluded because these represent a particularly weak research design with a serious threat to internal validity. Those who drop-out from a drug-court are likely to differ from those who remain in a drug court on a host of important variables, some observable and others not (see e.g., Mateyoke-Scrivner, Webster, Staton, Leukefeld, 2004).

The third and fourth criteria are largely self-explanatory. Simply put, eligible evaluations could use a wide variety of recidivism measures from official measures such as arrest, conviction, and re-incarceration to self-report measures of criminal offending. Last, all eligible evaluations had to report sufficient information for effect size calculation.

3.2 SEARCH STRATEGY FOR IDENTIFICATION OF RELEVANT STUDIES

The goal of the search strategy was to identify all evaluations, published or unpublished, meeting the above eligibility criteria. In order to achieve this objective, a multi-pronged search strategy was utilized. The search began by conducting a computerized keyword search of bibliographic databases. In particular, we conducted a search of the following databases: NCJRS, Criminal Justice Abstracts, Dissertation Abstracts, PsycINFO, Sociological Abstracts, Social Science Citation Index, Sciences Citation Index Expanded, Arts & Humanities Citation Index, Conference Papers Index, Ingentaconnect, C2 SPECTR, and CINAHL, as well as Google internet searches. The keywords used were: drug court (drug court*), DWI court (DWI court*), DUI court (DUI court*), evaluation, recidivism, re-arrest, and re-conviction. Each of the first three keywords was used in combination with each of the four latter terms.

We also searched for eligible evaluations by carefully reading retrieved studies and existing reviews of drug court research for unfamiliar evaluations. In particular, we reviewed the reference lists of existing reviews of drug court evaluations to identify eligible evaluations (Belenko 2001, GAO, 1997; 2005; Latimer, Morton-Bourgon, & Chretien, 2006; Marlowe, Festinger, Arabia, Croft, Patapis, & Dugosh, 2009; Shaffer, 2006, 2011) Likewise, many of the included evaluations reviewed prior drug court research. Unfamiliar studies referenced by eligible evaluations were also assessed for eligibility.

Further, we searched websites of several prominent research organizations. Specifically, we searched for relevant research reports on the following websites: NPC Research (http://www.npcresearch.com/); National Drug Court Institute (http://www.ndci.org/ndci-home/); the Drug Court Clearinghouse via American University's Justice Programs Office webpage (http://www1.spa.american.edu/justice-old/index.php); RAND Drug Policy Research Center (http://www.rand.org/multi/dprc/); The Urban Institute's website (http://www.urban.org/index.cfm); and, the University of Cincinnati's School of Criminal Justice publications page (http://www.uc.edu/ccjr/reports.html).

All studies that appeared to be eligible based on a preliminary review of the title and abstract were retrieved and closely scrutinized to determine final eligibility status. Specifically, we reviewed the title and abstract of each search result for clear evidence of ineligibility. Those studies that could not be ruled as ineligible based on the title/abstract review were retrieved for further assessment of eligibility. Retrieved studies were read closely to determine final eligibility status.

Note that the last search was conducted in August 2011. Given our extensive search we are confident that we have identified the overwhelming majority of eligible evaluations. It is possible, however, that studies made available close to August 2011 may have been missed, as there often is a lag between the availability of a study and its appearance in bibliographic databases and search engines.

3.3 DESCRIPTION OF METHODS USED IN THE COMPONENT STUDIES

The basic research design used in eligible evaluations was a two-group (treatment and comparison group) design with a post-release outcome measure of interest, such as post-release criminal arrest or drug use. The comparison groups employed in eligible evaluations were constructed using a variety of methodologies including historical comparisons, drug-involved probationers from nearby jurisdictions, drug-involved offenders eligible for the treatment program who chose not to participate, eligible offenders who did not participate for a variety of reasons (e.g., not referred, rejected by program administrators), and random assignment. The included evaluations also varied widely in regards to the degree to which they employed statistical controls (matching, covariate analysis, etc.) to reduce the threat of selection bias. Our coding forms were designed to capture these methodological variations.

3.4 CRITERIA FOR DETERMINATION OF INDEPENDENT STUDIES

Several types of statistical dependencies were evident in evaluations of drug court programs. One common dependency was created by multiple measures of criminal behavior (e.g., re-arrest, re-conviction, drug use) or multiple follow-up periods for the same indicator of criminal behavior ups (e.g., 6 months, 12 months). Another common dependency was produced by multiple studies reporting findings from the same sample of research participants.

The statistical methods detailed below required statistical independence. In other words, each evaluation had to use a unique research sample. We utilized several strategies to maintain the statistical independence. First, all evaluations (i.e., treatment/comparison contrasts) were cross-checked against one another to ensure that multiple studies reporting the results of the same evaluation do not contribute multiple estimates of program effects to any analysis. Second, in evaluations that report multiple measures of criminal behavior, rather than averaging these multiple outcomes, we applied a set of selection criteria that created three data sets of effect sizes, with a particular evaluation contributing only one effect size to each of the data sets. ⁵ In the first data set preference was given to effect sizes that: (1) were general (i.e., covered all offense types as opposed to being offense specific), (2) were based on arrest, (3) were dichotomous, and (4) followed sample members for 12 months. We preferred effect sizes meeting these criteria, because such effect sizes were the most commonly reported outcome measure. The goal here is to calculate one effect size from each evaluation meeting these criteria to maximize the comparability of recidivism measures across evaluations. If no such effect size was available, we selected the effect size that most closely matched these criteria. For example, property offenses were more general than violent offenses, effect sizes based on re-convictions were preferred over re-incarcerations, and effect sizes following sample members closest to 12 months were preferred over other effect sizes. Each independent evaluation contributed one, and only one, effect size to this “general recidivism” data set. This general recidivism data set served as the main data set in the analyses that follow.

We also created two other more specific data sets: one data set for measures of drug related recidivism (e.g., arrest or conviction on drug charges), and another data set for measures of drug use (via self-reports or drug tests). In creating these data sets we had considerably fewer effect sizes to choose from. When multiple effect sizes were available for any of these data sets, preference was given to effect sizes that: (1) were more general (e.g., encompassed multiple types of non-drug offending, instead of one specific type of non-drug offending), (2) were dichotomous, and (3) followed sample members for 12 months. If an evaluation did not report one of these specific types of outcomes, then that evaluation did not contribute to the particular data set.

3.5 DETAILS OF STUDY CODING CATEGORIES

The coding forms employed in this review are provided in Appendix B. These coding forms were structured hierarchically, in order to explicitly recognize the nested nature of effect sizes within studies. Any number of effect sizes could be coded from each evaluation using these forms [see Lipsey and Wilson (2001) for a discussion of this issue].

The coding forms captured key features of the nature of the treatment, research participants, research methodology, outcome measures, and direction and magnitude of observed effects. Two coders assessed each study. Discrepancies between coders were resolved by the lead author.

3.6 STATISTICAL PROCEDURES AND CONVENTIONS

An effect size was calculated for each evaluation contrast. We utilized the odds-ratio effect size, as this type of effect size is the most appropriate effect size for dichotomous outcomes, like our preferred recidivism measure (Lipsey and Wilson, 2001). When continuous outcome measures best fit the effect size selection criteria (see section 3.4), the standardized mean difference effect size was used. These effect sizes were coded in a manner such that positive effect sizes indicate the treatment group had a more favorable outcome than the comparison group (i.e., less recidivism or drug use).

Odds-ratio effect sizes and standardized mean difference effect sizes were combined using the methods developed by Hasselblad and Hedges (1995). Specifically, mean difference effect sizes were transformed to odds-ratios.

Our analyses of these effect sizes utilized the statistical approach outlined by Lipsey and Wilson (2001). In particular, we used the inverse variance method and assumed that the true treatment effects varied as a function of both measured (i.e., coded study features) and unmeasured differences between studies, that is, a random-effects model.

Our analyses employed Stata macro programs written by David B. Wilson.6 These macro programs calculated the random effects variance component discussed above and computed various statistics such as the overall mean effect and the homogeneity of effects statistic. We also used these macro programs to determine which study features were associated with observed study effects via meta-analytic analogs to analysis of variance and regression, assuming a mixed-effects model estimated via full-information maximum-likelihood (Raudenbush, 1994; Overton, 1998).

3.7 TREATMENT OF QUALITATIVE DATA

We did not include qualitative research in this systematic review. In future updates to this systematic review, however, we are open to suggestions from and collaboration with researchers specializing in such techniques.

4.1 DESCRIPTION OF ELIGIBLE STUDIES

Our search found 370 potentially eligible studies. We retrieved 365 of the potentially eligible studies for further review. (The remaining five studies cannot be located.) Of the retrieved studies, 181 were eligible for this systematic review. Many of the eligible studies, however, utilized overlapping samples or the same sample in initial and follow-up evaluations of one drug court. These 181 studies yielded 154 independent evaluations of drug court programs. These 154 eligible and independent evaluations form the sample for this systematic review.

Tables 1-3 provide descriptive information on the evaluations and drug courts examined in this review. Most of the drug court evaluations examined the effectiveness of adult drug courts (i.e., drug courts designed for adult illicit substance users). Ninety-two of the 154 evaluations (60%) assessed adult drug courts. Another 34 (22%) evaluations examined juvenile drug courts and the remaining 28 (18%) evaluations probed DWI (driving while intoxicated) courts.

All but eight of the eligible, independent evaluations examined U.S. drug courts. Four of the remaining evaluations examined adult drug courts in Australia. Two evaluations were of Canadian drug courts (Toronto and Vancouver). One evaluation assessed a juvenile drug court in New Zealand, and another examined an adult drug court in Guam. A majority of the evaluations have been conducted since 1999, enhancing the external validity of these evaluations to the contemporary criminal justice context, at least in the United States.

Overall, this body of literature is methodologically weak with few randomized studies of each type of drug court and only a modest number of rigorous quasi-experimental studies of adult drug courts and juvenile drug courts. This will be discussed in greater detail below in the context of the study findings.

The evaluations exhibited relatively little variation in terms of sample characteristics (see Table 3). Overwhelmingly, the samples used in these evaluations were composed predominantly of males. The vast majority of courts restricted eligibility to non-violent offenders. Approximately 25% of samples used in adult drug courts had minor criminal history; whereas, less than 10% of the samples in juvenile and DWI drug courts had minor criminal history, that is, they were mostly first-time offenders.

4.2 OVERALL MEAN EFFECTS BY TYPE OF DRUG COURT

We calculated effect sizes that measure drug courts' effects on three outcomes: (1) general recidivism (typically measured as re-arrest for any offense), (2) drug related recidivism (typically measured as re-arrest for a drug offense), and drug use (usually measured as self-reported drug use or via urinalysis), for each type of court. Table 4 displays the mean odds-ratio for each court type (adult, juvenile, and DWI) by outcome type (general recidivism, drug recidivism, and drug use). The forest plots for the general recidivism effects are shown in Figures 2 through 4. The forest-plots show a clear pattern of evidence favoring drug courts, with most studies observing effects favoring the drug court (88%, 70%, and 85%, for adult, juvenile, and DWI, respectively). The overall mean odds-ratio for the general recidivism measure is small to moderate in size and statistically significant for all three court types (mean odds-ratio of 1.66, 1.37, and 1.65, for adult, juvenile, and DWI, respectively). Relative to a 50% recidivism rate in the comparison group (a typical value), these odds-ratios translate into recidivism rates for the respective drug court groups of 37.6%, 42.2%, and 37.7%. Thus, on average participants in adult and DWI drug courts have recidivism rates approximately 12 percentage points lower than non-participants, while on average participants in juvenile drug courts have recidivism rates approximately 8% lower than non-participants.

The effects of these courts on drug related recidivism (i.e., drug related crimes) are very similar for adult and DWI drug courts with random effects odds-ratios of 1.70 and 1.65, respectively. However, for juvenile drug courts, the results on drug related recidivism outcomes were less encouraging. The mean odds-ratio was 1.06. In practical terms, this odds-ratio is essentially null, or a no-difference effect. Thus, the current evidence raises the possibility that juvenile courts are not effective in reducing drug related crime or that any effect that is produced is small.

Surprisingly few evaluations assessed the effect of drug court participation on measures of actual drug use (i.e., urinalysis or self-reported drug use). We found only nine independent evaluations that reported useable post-program entry measures of drug use for both drug court participants and non-participants. Four of the nine effect sizes assessed the effect of participation in adult drug court programs, three were from evaluations of juvenile drug courts, and the two remaining effect sizes were from evaluations of DWI drug courts. For each type of court, the average effect size was positive, indicating reduced drug use for drug court participants in comparison to non-participants. However, because of the small number of drug use effect sizes, the average effect size for each type of court was not statistically significant.

Although the evaluation of all three types of drug courts indicate that drug court participants have, on average, lower rates of general and drug recidivism than non-drug court participants, these findings need to be interpreted within the context of the methodological rigor of the studies. As discussed below, this body of literature is generally weak methodologically.

4.3 ROBUSTNESS OF FINDINGS TO METHODOLOGICAL WEAKNESSES

Evaluations were placed into four categories: (1) weak quasi-experiments, (2) standard quasi-experiments, (3) rigorous quasi-experiments, and (4) randomized experiments. Rigorous quasi-experiments typically used subject-level matching on key variables or propensity score matching. Standard quasi-experiments typically used either a historical comparison group that met drug court eligibility criteria constructed from archival data or a group of offenders who were eligible but not referred to the drug court program. The critical feature here is that the participants did not self-select into the drug court or comparison condition. Weak quasi-experimental designs typically involved comparing drug court clients to drug offenders who were eligible for participation in a drug court but declined participation (“refusers”) or were referred to the drug court but were rejected by drug court administrators (“rejects”). Such designs have questionable internal validity because refusers and rejects are likely to differ on factors like pre-treatment motivation, perceived seriousness of drug problem, and self-efficacy, among many other potentially important factors. It's important to note that evaluations that used refusers or rejects as the comparison group but included efforts to minimize selection bias (e.g., controlled for many variables in the analyses) were given higher ratings depending on the nature of the efforts to minimize selection bias.

Tables 5-7 display the mean effect size by methodological features for general recidivism and drug recidivism. Focusing on adult drug courts, we find that the magnitude of the estimated effects of participation in adult drug courts generally declines as methodological rigor increases, but this relationship is not statistically significant. In particular, evaluations rated as weak or standard quasi-experiments found sizeable and statistically significant effects of drug court participation on measures of recidivism. These evaluations, however, are plagued by a substantial threat of selection bias. The clearest evidence of the effectiveness of adult drug courts is provided by evaluations rated as strong (rigorous) quasi-experiments. The mean effect sizes for these evaluations are of a meaningful size and statistically significant.

Of concern is the smaller average effects found in the three most rigorous, experimental evaluations of adult drug courts. In fact, for both general recidivism and drug recidivism, the mean odds-ratio for experimental designs is not statistically significant across the three experimental evaluations, and near the null value for drug related recidivism. These findings from these studies deserve greater exploration. The three randomized experimental evaluations of adult drug courts included in this synthesis evaluated: (1) a drug court in Maricopa County (Deschenes, Turner, & Greenwood, 1995; Turner, Greenwood, Fain, & Deschenes, 1999); (2) Baltimore City's drug court (Gottfredon, Najaka, & Kearley, 2003; Gottfredon, Najaka, Kearley, & Rocha, 2006); and, (3) a drug court in New South Wales, Australia (Shanahan, Lancsar, Haas, Lind, Weatherburn, & Chen, 2004). All three evaluations reported findings on the effectiveness of drug court participation on measures of general and drug related offending in the first 12 months after program entry, which are the effects included in our primary analyses. The results for these experimental evaluations are inconsistent (heterogeneous), with two of the three evaluations finding modest positive results (odds-ratios of 1.65 and 1.82), and one evaluation (Deschenes et al., 1995) with a near null effect (1.06) on the general recidivism outcome measure, and two of the three studies finding small negative effects on the drug recidivism outcome measure (Deschenes et al., 1995, and Gottfredson et al., 2003). Thus, the finding that experimental evaluations of adult drug courts do not collectively find statistically significant reductions in recidivism is driven by inconsistent results across the three evaluations and the small number of experimental evaluations (i.e., low statistical power), which is evidenced by the rather large confidence intervals around the mean effect sizes.

All three evaluations have unique characteristics; however, the Maricopa County program had the unusual characteristic of comparing the drug court participants to a control group of offenders involved in a drug-testing program. In fact, the drug-testing control group was drug tested more often than the drug court group—a highly unusual finding in this body of research. Consequently, the evaluation of Maricopa County lacked a key feature of drug court evaluations, a treatment group that received more frequent drug testing that the comparison group. This fact, among others, makes this evaluation problematic. An important issue concerns the effect of this unique evaluation on the results reported in Table 5. We find that if this evaluation is removed for the analysis, the mean odds-ratio for experimental evaluations on the general recidivism outcome measure is 1.73 (95% C.I. of 1.18 to 2.53), which is statistically significant and larger than the effect for the rigorous quasi-experimental designs. The mean odds-ratio for the drug recidivism measure remains essentially unchanged (1.05, with a 95% confidence interval of 0.69 to 1.60). Another complication with the evaluation of the Maricopa County program is that the results changed substantively in a follow-up evaluation that reported outcomes three-years after program entry (Turner et al., 1999). In this subsequent evaluation, participants in the Maricopa County drug court program had significantly less recidivism (general and drug) than the control group. If the effect sizes from this subsequent evaluation are used instead of those from the original evaluation, then the mean odds-ratio for experimental evaluations is 1.65 (95% C.I. of 1.25 to 2.18) for general recidivism, which is statistically significant and virtually identical to the mean effect size for all evaluations of adult drug courts, and mean odds-ratio for drug recidivism is 1.19 (95% C.I of 0.82 to 1.73), which is not statistically significant. Thus, the results of the Maricopa County evaluation materially affect the size and statistical significance of the mean odds-ratio for general recidivism, but not drug recidivism.

We draw three conclusions from the above analyses. First and foremost, all three experimental evaluations of adult drug courts provide evidence of these courts' effectiveness in reducing recidivism. Two of the three evaluations find recidivism reductions in the first year after program entry, and the remaining evaluation finds recidivism reductions at three years post-program entry. Second, because the vast majority of adult drug court evaluations, even the most rigorous evaluations, find moderate reductions in general recidivism, we believe that the evidence indicates that adult drug courts reduce recidivism. Third, the mean odds-ratio effect size measuring these courts effect on general recidivism appears to be approximately 1.65, which translates into an average recidivism rate of 38% for drug court participants, if we assume a 50% recidivism rate for non-participants.

In regards to evaluations of juvenile drug courts, we find that these courts have small effects on recidivism, especially in methodologically rigorous evaluations. The strongest evidence of the effectiveness of juvenile drug courts comes from weak quasi-experimental evaluations, as in these evaluations the general recidivism mean odds-ratio is relatively large (1.85) and statistically significant. However, the general recidivism mean odds-ratios are considerably smaller in evaluations with higher levels of methodological rigor. For standard quasi-experimental evaluations, the mean odds-ratio is 1.32 (95% C.I. of 1.07 to 1.62). The mean odds-ratios for strong (rigorous) quasi-experimental and experimental evaluations are similar in magnitude 1.32 and 1.22, respectively; neither is statistically significant. However, if the rigorous quasi-experimental and experimental evaluations are combined, the mean effect size is 1.28 (95% C.I. of 1.03 to 1.61), which is statistically significant. These findings indicate that evaluations of juvenile drug courts reduce general recidivism, but the magnitude of these effects is smaller than that of adult drug courts. For drug related recidivism, juvenile drug courts' strongest effects are found in more rigorous evaluations, and in these evaluations the magnitude of the effect is similar to the general recidivism means odds-ratio (approximately 1.30). Thus, the most rigorous evaluations of juvenile drug courts indicate that these courts have small effects on recidivism and the magnitude of these effects is approximately an odds-ratio of 1.30, which translates into a 43.5% recidivism rate for drug court participants, if we assume a 50% recidivism rate for non-participants.

It is important to note that three evaluations of juvenile drug courts used experimental designs. Two of these evaluations assessed two different cohorts of participants in the Summit County (Ohio) Juvenile Drug Court (Dickie, No Date). These two evaluations had large attrition problems (both had more than 50% total attrition and one of the cohorts had significant differential attrition). Because of these attrition problems, we rated these two compromised experiments as strong quasi-experiments. Thus, the results presented in Table 6 for experimental designs are for the single high-quality randomized trial. This study found positive effects, albeit not statistically significant, that are only slightly smaller than the overall effects for adult drug courts (i.e., an odds-ratio of 1.39 for general recidivism and 1.38 for drug recidivism). If the two compromised evaluations are rated as experiments, our findings are essentially unchanged: the mean odds-ratio for general recidivism is 1.44. While the mean effect size is not statistically significant, it is important to note that the effect sizes from these three evaluations are all positive, indicating less recidivism for juvenile drug court participants. These findings continue to support the conclusion that juvenile drug courts have small effects on recidivism.

The substantive findings regarding the effectiveness of DWI drug courts are similar to those for adult drug courts. Just as with evaluations of adult drug courts: (1) the largest effects of DWI drug courts are found in methodologically weak evaluations, (2) collectively, experimental evaluations find small and non-statistically significant mean odds-ratios for general and drug related recidivism; and, (3) the mean odds-ratios for experimental evaluations are greatly influenced by one evaluation that found a negative effect of participation. In regards to the first point, the mean odds-ratio is moderate and statistically significant for the three quasi-experimental design categories, but relatively small and non-statistically significant for the four evaluations in the experimental category (1.27 with 95% C.I. of 0.87 to 1.85). Thus, quasi-experimental evaluations yield substantively different findings than experimental evaluations of DWI drug courts. In regards to the second point, experimental evaluations find a small and non-statistically significant difference in general and drug related recidivism between participants and non-participants. And finally, additional analysis of the four experimental DWI drug court evaluations reveals that one such experimental evaluation conducted by MacDonald, Morral, Raymond, & Ebner (2007) heavily influences the mean odds-ratio for the experimental evaluations. While the three other experimental evaluations of DWI courts found positive odds-ratios on the general recidivism measure ranging from 1.39 to 2.25, MacDonald et al.‘s evaluation found a negative effect (odds-ratio of 0.73). If MacDonald et al.‘s evaluation is omitted from the general recidivism analysis, then the mean odds-ratio becomes 1.58 (95% C.I. of 0.99 to 2.54) and has a p-value of 0.057. Similarly, if this evaluation is omitted from the drug related recidivism analysis, the mean odds-ratio becomes 1.43 (95% C.I. of 0.83 to 2.50) with a p-value of 0.194. Thus, the MacDonald et al. evaluation is highly influential on the results of the experimental evaluations, especially the general recidivism analysis.

The evidence presented above finds considerable evidence of the effectiveness of drug courts, but the strength of this evidence varies by type of court. In regards to adult drug courts, over 90 independent evaluations have been conducted and the overwhelming majority of these evaluations find that drug court participants have less recidivism than non-participants. Further, experimental evaluations of these courts also consistently find sizeable reductions in recidivism. Thus, the evidence indicates that adult drug courts are effective in reducing recidivism. Likewise, we characterize the evidence as cautiously supporting the effectiveness of DWI drug courts, because while quasi-experimental evaluations find strong and consistent indications that these programs reduce general and drug related recidivism, randomized experimental evaluations find a small, non-statistically significant reduction in recidivism. Yet, the findings from experimental evaluations of DWI drug courts are ambiguous in that the majority of these evaluations find positive effects but a single, influential evaluation with negative findings heavily influences the mean effect. Clearly, only additional evaluations using experimental methods can definitively resolve the remaining ambiguity surrounding the effectiveness of DWI drug courts general effectiveness. Evaluations of juvenile drug courts, especially more rigorous evaluations, consistently indicate that these courts have relatively small effects on recidivism.

4.4 DRUG COURTS' LONG-TERM EFFECTS

An important issue in drug court research is whether the effects last long-term. Assessing drug courts' long-term effect, however, is challenging. There are two interrelated issues. The first is whether the observed pattern of positive results reflects a suppression effect. Many of the outcomes are examined for recidivism during the course of drug court participation. It is possible that drug courts suppress offending behavior while someone is active in the program but that this effect disappears post-program once behavioral contingencies are removed. The second issue is simply whether observed effects continue long-term, such as three years post-program.

We examined the first issue of suppression by coding whether the recidivism tracking period overlapped: (1) completely, (2) partially, or (3) not at all with the period of drug court participation. If drug courts' effects on recidivism are limited to the period in which participants are active in the court, then the mean effect size should be largest when the treatment and recidivism tracking periods overlap completely and the mean effect size should decrease as the amount of overlap between the treatment and recidivism tracking periods decreases. Table 8 displays the mean effect size for each of these categories and shows that the effects of adult drug courts remain post-program. That is, the positive results do not appear to be simply a temporary suppression effect.

We examined the second issue by computing mean effect sizes for different follow-up periods. Recall that evaluations most commonly reported recidivism rates 12 months after drug court entry (or termination). When effect sizes measuring recidivism at multiple time points were available, we preferred effect sizes based on this most common time period (i.e., 12 months) to facilitate between study comparisons. However, not all evaluations measured recidivism at 12-months and some evaluations reported recidivism at multiple time-points (e.g., 12, 24, and 36-months). Consequently, there is variation both within and between studies in the length of recidivism tracking period. We exploit both sources of variation. Between study variation was examined by calculating the mean effect size by length of recidivism tracking period. These findings (Table 8) show a roughly stable mean odds-ratio for the different follow-up periods. A complication with this analysis is the possible confounding of evaluation features with follow-up length. To address that issue, we also examined within study variation by analyzing the subset of evaluations that reported results at both the 12 and 24 month follow-up period (21 studies) and the subset of evaluations that reported results at the 12, 24, and 36 month follow-up period (8 studies). As shown in Table 8, adult drug court effects remain remarkable stable over time from 12-months through 36-months.

The analyses presented above support the conclusion that any effect adult drug courts have on recidivism are not limited to the short-term. Rather, the available research suggests that drug court participants have reduced recidivism during and after drug court treatment, and these effects appear to last at least three years post-drug court entry.

4.5 FEATURES OF THE DRUG COURT

The various drug courts examined by this collection of evaluations vary in potentially important ways (see Tables 2 and 3). We were guided by Longshore et al.'s conceptual framework for understanding differences across drug courts.

Longshore et al's framework has five dimensions: leverage, population severity, program intensity, predictably, and rehabilitative emphasis. We were able to code measures tapping the first three dimensions. We were unable to code measures of the two latter concepts, because few evaluations reported the kinds of information necessary to measure these concepts. While we were able to code measures of leverage, population severity, and program intensity, many of our measures contain a large amount of missing data because evaluations often failed to provide relevant information (esp. regarding program intensity).

Longshore and colleagues argued that drug courts in which participants face greater consequences if they fail to meet program requirements have greater leverage. For example, they contend drug courts that process cases using the post-plea method have greater leverage because offenders have been convicted and face immediate sentencing upon failure in drug court. Likewise, drug courts that dismiss charges or expunge convictions have greater leverage because such incentives are more attractive motivators than reductions in charges/sentences. Therefore, we coded courts' method of case disposition (pre-plea, post-plea, or mixed) and what happens to the charges/sentences upon graduation (dismissed/expunged or not dismissed/expunged). Again evaluations often failed to report relevant information. However, when evaluations did report such information, post-plea case processing was most common (see Table 2). Further, most adult drug courts dismissed charges upon graduation, but dismissal was less common in juvenile and DWI drug courts.

We found mixed support for the relationship between leverage and outcomes (Table 9). As anticipated, drug courts that dismiss/expunge charges upon graduation had higher mean odds-ratios on both measures of recidivism. This difference is statistically significant for the drug recidivism measure, but is non-significant for the general recidivism measure. Counter to our expectations, however, drug courts that predominantly used post-plea case processing did not have greater reductions in recidivism than other courts.

We coded multiple measures designed to tap program intensity, another aspect of Longshore et al.'s framework (see Table 2). Moderator analyses of the relationship between coded measures of program intensity and effect size generally did not generally support the hypothesis that more intense programs are associated with greater reductions in recidivism (Table 9). In fact, most of these relationships reveal that the mean odds-ratios were substantively and statistically similar across the categories of the coded program intensity measures. One notable possible exception to this conclusion is the relationship between number of status hearings and effect size. The adult drug court analysis finds that courts with more than two status hearings in first treatment phase had larger effects of drug related recidivism than other courts. Other than the number of status hearings, these findings suggest that drug courts with greater program intensity are no more effective in reducing recidivism than other courts.

We replicated these analyses for juvenile and DWI drug courts (see Tables 10 and 11). These analyses were hindered by the relatively small number of evaluations of these programs and by missing data. Few differences were found. Yet, it is important to note that these analyses found that juvenile drug courts with more frequent status hearings had larger effects on general recidivism than other courts—a finding that is substantively similar to that found in the analysis of adult drug court evaluations.

Perhaps most controversially, Longshore et al. hypothesize that drug court programs that serve less severe populations, in terms of criminal history and substance abuse problems, are more successful in reducing recidivism. This hypothesis is controversial as it directly contradicts a core element of Andrews et al.‘s (1990) principles of effective intervention. One of Andrews et al.‘s principles is that more effective programs serve more severe populations, and this “risk principle” has found empirical support (see e.g., Andrews & Bonta, 1992). To test the relationship between population severity and program effectiveness (i.e., effect size), we coded two measures of population severity. The first measure assessed whether offenders with violent criminal history were allowed into the program. The second measure concerned each sample's extent of criminal history. Many drug courts also limit eligibility to offenders without extensive prior convictions. We inductively coded a measure flagging evaluations that used samples with limited criminal history. More specifically, we took notes on information relating to eligibility restrictions on criminal history and/or descriptive statistics concerning criminal history. After all evaluations were coded, we read over these notes and distinguished samples with relatively minor criminal history. For example, if 70% of the sample of drug court participants had no prior convictions or less than 3 prior arrests, then this sample was coded as having minor criminal history.

Table 12 reports the mean odds-ratios by coded sample characteristics from evaluations of adult drug courts. Analyses of the general recidivism effect sizes support Longshore and colleagues prediction. Specifically, samples that included only non-violent offenders had statistically larger mean odds-ratios on the general recidivism measure than samples that included violent offenders. Similarly, samples with minor criminal history had larger mean odds-ratios than evaluations based on samples with more criminal history; however, this difference was not statistically significant. These findings were not replicated in the analysis of the drug related recidivism outcome measure. Here there were no differences in the mean effects odds-ratios on either of the measures of population severity. Thus, these findings are more supportive of Longshore et al.‘s hypothesis than Andrews et al.‘s risk principle.

Again, we replicated these analyses for juvenile and DWI drug courts (see Tables 13 and 14) and no meaningful relationships were between effect size and population severity. We also coded important drug features that are not directly related to Longshore et al.'s framework. One such measure is program graduation rate (see Tables 2 and 9). Here we found that most drug court participants do not successfully complete the program. The median graduation rate for adult, juvenile, and DWI drug courts are 39%, 47%, and 62%, respectively. Interestingly, for all three types of drug courts, we find a strong, non-linear relationship between graduation rate and both general and drug related recidivism. Specifically, Tables 9, 10, and 11 indicate that courts with graduation rates between 26% and 50% had substantively smaller effects on general and drug related recidivism than courts with either higher or lower graduation rates. Additional analyses (not shown in the tables) indicate that all of these differences are statistically significant (each has a p-value less than 0.01). Thus, our results indicate that courts with graduation rates between 26% and above 50% had larger effects than courts with graduation in between these percentages. It is unclear whether this reflects a feature of the drug court itself or of the client pool.

Another potentially important measure is the network of treatment providers utilized by the drug court. Some research suggests that courts employing a single treatment provider are more likely to use cognitive-behavioral programs (Peyton & Gossweiler, 2001), which have been shown to be relatively effective (MacKenzie, 2002, 2006). Additionally, the use of a single treatment provider may indicate stronger lines of communication between the court and the treatment providers. As such, courts using a single treatment provider may be more effective in reducing recidivism than other courts. Despite these predictions, drug courts that utilize a single treatment provider had similar mean effect sizes as other courts.

4.6 ADDITIONAL SENSITIVITY ANALYSES

The majority of the evaluations included in this review were unpublished technical reports produced by government or private research entities. Such a large proportion of unpublished evaluations reduced the likelihood of publication bias affecting our estimates of drug courts' effectiveness. As shown in Table 5, the results for published and unpublished evaluations of adult drug courts were roughly similar. This finding is also true for juvenile and DWI courts (see Tables 6 and 7).

Other forms of publication selection bias, such as outcome selection bias, may be present. To assess whether our estimates were upwardly biased due to some form of publication bias, we performed the Duval and Tweedie trim-and-fill analysis for each court type. This method assumes that in the absence of publication bias, the scatterplot between effect size and standard error of the effect size will have a funnel shape and augments the data to achieve this shape. Using this method, the distribution of general recidivism effect sizes for adult drug court was filled with 23 effect sizes, which reduced the overall random effects mean odds-ratio from 1.66 to 1.34. The latter remained statistically significant (z = 5.18, p < 0.01). The distribution of drug recidivism effect sizes for adult drug courts did not require filling. For juvenile drug courts, the distribution of general recidivism effect sizes did not require filling but the distribution of drug recidivism was augmented with one additional effect size, which reduced the random effects means odds-ratio from 1.06 to 1.01 (neither was statistically significant). For the DWI drug courts, five effect sizes were added to the general recidivism distribution, which reduced the mean odds-ratio to 1.53 (z = 3.57, p < 0.01); two effect sizes were added to the drug related recidivism distribution, which reduced the mean odds-ratio to 1.57 (z = 4.59, p < 0.01). Thus, the conclusions remain robust even under the trim-and-fill model that tends to over-fill when there is substantial heterogeneity, as was the case here.

We also performed sensitivity analyses to determine the effect on our results of allowing a few authors to contribute many effect sizes to our analyses. Specifically, we flagged authors who contributed 10% or more of the effect sizes to any of the court specific analyses, and then re-ran the analysis without these effect sizes. For adult drug courts, only NPC research contributed 10% or more of the effect sizes (k = 15). While the effect sizes from NPC had a larger mean odds-ratio effect size (1.94) than other evaluations (1.63), this difference was not statistically significant and the mean odds-ratio effect size for the other evaluations is virtually identical to the mean effect size with the NPC evaluations included. We performed similar analyses with the juvenile and DWI drug court effect sizes. These sensitivity analyses revealed that our results are substantively unchanged by multiple effect sizes from the same author(s).

Last, we assessed the applicability of our findings beyond the United States by examining the eight international evaluations. Seven of the eight evaluations examined adult drug courts (1 in Guam, 2 in Canada, and 4 in Australia). Six of these seven evaluations had positive general recidivism effect sizes, four of which were moderate in size (odds ratio greater than 1.60), and the mean general recidivism odds-ratio is 1.62 with a 95% confidence interval of 1.11 to 2.36, which is statistically significant. Notice that this mean general recidivism odds-ratio is virtually identical to the mean odds-ratio reported above for all adult drug court evaluations (1.66). Only one international evaluation of a juvenile drug court met our eligibility criteria; this evaluation assessed a juvenile drug court in New Zealand (Searle & Spier, 2006). This evaluation found a negative effect indicating that the juvenile drug court did not reduce recidivism. These international evaluations mirror our findings on adult and juvenile drug courts. Thus, our findings appear to accurately reflect existing evaluations of drug courts outside the United States.