PROTOCOL: Family treatment drug courts for improving parental legal and psychosocial outcomes

1.1 The problem, condition or issue

A substantial portion of parents involved in child welfare systems have co-occurring substance abuse issues (Laslett, Room, Dietze, & Ferris, 2012; Miller, Orellana, Johnson, Krase, & Anderson-Nathe, 2013; Williams, Tonmyr, Jack, Fallon, & MacMillan, 2011; Young, Boles, & Otero, 2007). In addition, child welfare cases characterized by parental substance abuse tend to result in more detrimental outcomes for families, than cases without parental substance abuse issues (Brook & McDonald, 2009; Connell, Bergeron, Katz, Saunders, & Tebes, 2007; Smith, Johnson, Pears, Fisher, & DeGarmo, 2007). In the United States, the high rate of parental substance abuse amongst child welfare cases transitioned to judicial settings (Miller, 2004) led to adaptation of the adult drug court model into a Family Treatment Drug Court to deal with child welfare cases characterized by parental substance abuse. Since their 1994 inception in the United States (US), Family Treatment Drug Courts (FTDCs) have increased in popularity over the last 15 years (Bruns, Pullmann, Weathers, Wirschem, & Murphy, 2012; Fay-Ramirez, 2015). For example, US FTDCs have increased from 2 in 1994 to 495 in 2018 (Lemus & Ritcher, 2018) and the model has been recently implemented in the United Kingdom (Bambrough, Shaw, & Kershaw, 2014; Harwin et al., 2018) and Australia (Children's Court of Victoria, 2019; Marshall, 2015). This trend is, in part, driven by increasing recognition that individuals that come through the criminal, juvenile and civil (family) court system often have co-occurring issues such as mental health or addiction issues that need to be addressed in order to motivate behavior change in the interest of children, or the parent themselves (Fay-Ramirez, 2015; Tiger, 2012). For child welfare cases, parental substance abuse is seen as the problem that hinders the establishment of a stable family environment that would enable the child's return to parents’ care. Thus, the primary goal of FTDCs is to treat the parental legal and psychosocial issues in child welfare cases, with a key aim being to reunify families, achieve permanent placements for children in a timely manner, and address substance abuse issues among parents (Gifford, Eldred, Vernerey, & Sloan, 2014).

Although FTDCs are becoming more popular around the world, the state of evidence for the impact of FTDCs on parent-level outcomes, beyond parent–child reunification, is currently unclear. Some research suggests that the FTDC model promotes better long-term outcomes for parents and families over their traditional courtroom counterparts (e.g., Picard-Fritsche, Bryan, Kralstein, & Farley, 2011). However, other research is less equivocal. For example, Lloyd (2015) argues that parents are less successful within the FTDC model and appear to be at higher risk of having their parental rights terminated than if their case is processed in a traditional dependency court. In addition, scholars have raised concern over whether studies showing positive outcomes have been driven by the FTDC therapeutic model of justice, or the extent of surveillance used by the FDTCs to monitor parents and families (Marlowe, DeMatteo, & Festinger, 2003; Tiger, 2012). Moreover, existing reviews of FTDCs lack methodological rigor, fail to integrate the full range of parental legal and psychosocial outcomes, or tend to focus on child outcomes (see “Existing Reviews”). Although a focus on child outcomes is warranted, changes in parent-level factors are critically important for generating changes in child outcomes. Without a comprehensive understanding of the effect of FTDCs on parental legal and psychosocial outcomes, the full impact of FTDCs on vulnerable families with both child welfare and parental substance abuse issues remains unclear. Therefore, a methodologically rigorous review and synthesis of the growing number of FTDC impact evaluations is required to thoroughly understand whether FTDCs are effective for improving parental legal and psychosocial outcomes.

1.2 The intervention

FTDCs are an example of a growing number of “specialized” or “problem orientated” courts that use a nonadversarial approach for substance abusing parents who have ongoing child welfare cases being monitored by the judicial system (Gifford et al., 2014; Lloyd, 2015; Picard-Fritsche et al., 2011). The model originated in the United States in 1994 and sit within the civil jurisdiction, whereas in other countries, FTDCs sit within the family court system (e.g., Australia and United Kingdom (Levine, 2012). ¹ The primary goal of FTDCs is to reunite children with their parents as well as achieve parental sobriety by using (a) an extended treatment team of mental health practitioners, child advocates, attorneys and social workers; (b) regular in- and out-patient drug/alcohol treatment; (c) regular drug and alcohol testing; and (d) regular court hearings to maintain supervision by the treatment team (Edwards & Ray, 2005; Haack, Alemi, Nemes, & Cohen, 2005; Chuang, Moore, Barrett, & Young, 2012).

FTDCs differ from a traditional pathway in a number of ways (often referred to as Traditional Dependency Court). Traditional dependency ² courts have jurisdiction for all child protection matters regardless of whether the parent has co-occurring substance abuse issues. Core differences between the FTDC model and the traditional child welfare model, involve increased monitoring in FTDC (as often as weekly) versus traditional court (every 6 months), increased contact with a treatment team in FTDCs, and increased involvement of the treatment team in case management and inpatient/outpatient substance abuse treatment (Fay-Ramirez, 2015; Gifford et al., 2014; Picard-Fritsche et al., 2011). Figure 1 shows typical FTDC processes in comparison to typical traditional court pathways and Appendix A provides a description of common graduation and components of an 18-month FTDC program.

OPEN IN VIEWER

1

Case processing for Family Treatment Drug Courts (FTDCs) and traditional Dependency Courts

FTDCs typically involve in- or out-patient treatment of drug and alcohol addiction, monitoring and supervision whist in the FTDC program, incentives and sanctions for treatment compliance and other prosocial family behavior, mental health counseling and parenting assistance (Fay-Ramirez, 2015; Harden, Harper, & Popovits, 2018). In addition, FTDCs often use a range of consequences in order to maintain compliance with FTDC orders such as—at the most extreme end—criminal sanctions or limited parental rights to a child (Fay-Ramirez, 2015; Harden, Harper, & Popovits, 2018).

Existing reviews and qualitative descriptions of FTDCs show a high degree of consistency in their description of the intervention (Ashford, 2004; Fay-Ramirez, 2015; Worcel, Furrer, Green, Burrus, & Finigan, 2008). Parents with child protection concerns and co-occurring substance abuse issues can volunteer for an FTDC program. Once accepted into the FTDC, parents typically begin a series of assessments that determine what kinds of resources parents and their children may need. This will include inpatient or outpatient treatment, but also a wide range of other services to help improve the family environment, from employment assistance to parenting training. Parents will come to court every two weeks for a “review hearing” where the judge and treatment team will review the parents’ progress and make decisions about problems or issues that arise for parents and their children. These review hearings serve as an opportunity for the treatment team, the judge and the parent to interact about each case. Review hearings continue through the parents’ time in FTDC until they “graduate” from the program having met all program requirements. It is also a key model component that the treatment team meet informally outside of court to solve problems that arise and discuss each case prior to court review hearings. This integrated courtroom model is typically (a) nonadversarial, (b) makes decisions on a case-by-case basis although it relies on a systematic method of applying behavioral modification techniques and (c) focuses on rehabilitation rather than punishment (Fay-Ramirez, 2015).

FTDCs involve scaffolded program goals with incentives for good behavior and less supervision over time. FTDC programs vary in length but typically last for 18-months. This 18-month time frame is usually dictated by legislation, which determines how much time the court has to find a permanent placement for the child who is subject of the child protection concern. For example, US federal policy dictates that a reunification hearing is necessary at 12 months and termination of parental rights if a child is in foster care for 15 out of 22 months. This placement can be with the parent or through adoption or legal guardianship (previously long-term foster care). Graduation requirements from FTDCs most often consist of a range of goals having been reached by the parent. These goals include a period of sobriety from substance abuse, consistent engagement with counseling or other support services, suitable housing for the family, employment or other goals determined on a case-by-case basis by the FTDC treatment team over the course of the program (Fay-Ramirez, 2015). However, graduation requirements are not entirely subjective and are described in FDTC participant manuals.

The FTDC intervention is based on the recognition and understanding that the majority of cases in the child protection system involve multi-layered and complex issues (Gifford et al., 2014). The FTDC is thought to (a) reunify families and (b) improve substance abuse issues by integrating treatment, social support and social resources into a nonadversarial, rehabilitation-based court program. These options for substance abuse treatment and other support services are offered in the traditional courtroom model, however, compliance with court mandated treatment or resources is often low and considered to contribute to the high number of families with repeated contact with the child protection system (Fay-Ramirez, 2015). FTDCs are thought to improve compliance by using the judge and treatment team to monitor and compel compliance.

Although FTDCs, and Drug Courts more broadly, precede the concept of Therapeutic Jurisprudence (TJ), TJ has been used to understand and justify FTDCs (Fay-Ramirez, 2016; Nolan, 2009). In its simplest terms, TJ is the understanding that the law and experiences of the law can have potentially therapeutic or anti-therapeutic effects (Wexler & Winick, 1991). In the case of FTDCs, the nonadversarial nature of the program, the case-by-case decision making, and the understanding that the focus should be on rehabilitation are thought to underpin increased compliance with mandated court orders involving treatment and services.

Designed to offer more rehabilitative pathways to prosocial behavior for offenders with co-occurring drug and/or mental health issues, the drug court model (including FTDCs) deviate from the traditional courtroom model by offering more discretion for judges, flexible treatment options, and increased monitoring of the offender (Levine, 2012). Drug Courts, including FTDCs, are thought to provide increased contact and individualized treatment to offenders in order to maximize positive behavioral outcomes, including desistence from crime, family reunification, sobriety and mental health stability/treatment.

Goldkamp, White, and Robinson (2001) provide a framework for understanding how Drug Courts (including FDTCs) increase positive outcomes for its clientele.

Figure 2 provides a logic model that describes the characteristics of the Drug Court mode that help promote positive outcomes including sobriety, mental health stability, desistance from crime and resolution existing criminal justice warrants/cases, and family reunification. Offender attributes often determine whether parents/offenders agree to FTDC/Drug Court supervision, but they also shape parental outcomes, mental health issues, employment opportunities, available support networks and history of drug abuse and treatment, which all relate to increasing likelihood of reoccurring contact with the criminal justice system and the child welfare system (Miller et al., 2013). Goldkamp et al. (2001) describe Drug Court characteristics as (a) deterrent; and (b) rehabilitative, reflecting the need to provide incentive to submit to the Drug Court program, and provide flexible services matched to the individual needs of the parent/offender. Deterrent drug court elements include increased monitoring well above what would be expected in a traditional court model. This typically includes increased contact with the supervising judge, frequent drug testing, required and frequent court attendance, and sanctions when program rules are violated. Rehabilitative elements of drug courts include frequent contact with the judge (builds rapport), drug and mental health treatment, opportunities to access other services such as parenting classes, educational services, and use of the courtroom workgroup as social support—a core element of the therapeutic approach. Parents and offenders are also provided with rewards for good behavior and reaching sobriety milestones. These deterrent and rehabilitative elements of the drug court together are thought to produce more positive outcomes for offenders/families.

OPEN IN VIEWER

2

Family Treatment Drug Court logic model

There are several existing reviews that either focus entirely FTDC or capture one or more FTDC impact evaluation studies. Lloyd (2015) conducted a systematic review to examine the impact of FTDCs on children's time in foster care and reunification and conducted a qualitative synthesis of FTDC evaluations that used research designs with a comparison group. Wittouck, Dekkers, De Ruyver, Vanderplasschen, and Vander Laenen (2013) aimed to synthesize existing evidence for drug treatment courts on substance use more generally and provide a qualitative synthesis that includes FTDC studies. Eldred and Gifford (2016) used systematic search and screening to identify peer-reviewed US research that examines the use of legal approaches for addressing child maltreatment. The authors provide a brief narrative synthesis with cites to some existing FTDC evaluation studies. Other authors provide traditional narrative reviews of FTDC literature that capture studies assessing the impact of FTDCs on a range of different child and parent outcomes (Choi, 2012; Harrell & Goodman, 1999; Marlowe & Carey, 2012).

Lloyd (2015) and Zhang, Huang, Wu, Li, and Liu (2019) provide the most methodologically robust reviews, yet these reviews still either only focus on child outcomes or the synthesis only includes studies that reported statistically significant outcomes (Lloyd, 2015) or a restricted timeframe (Zhang et al., 2019). While Wittouck et al. (2013) and Eldred and Gifford (2016) include some studies with parent-level outcomes, their reviews are subject to bias by only searching one database (Wittouck et al., 2013) or excluding research not published in peer-review outlets (Eldred & Gifford, 2016). Moreover, although remaining narrative reviews are useful for gauging the breadth and characteristics of existing Family Treatment Drug Court impact evaluations, their synthesis approach is inadequate for providing reliable and precise estimates of intervention impact (Borenstein, Hedges, Higgins, & Rothstein, 2009; Littell, Corcoran, & Pillai, 2008).

These methodological quality issues and gaps in content coverage limit the conclusions that can be drawn about the effectiveness of Family Treatment Drug Courts for improving parental legal and psychosocial outcomes. Therefore, this review will both update and enhance the existing body of reviews by providing a synthesis of the evidence for the effectiveness of Family Treatment Drug Courts on legal and psychosocial outcomes for parents. This is important for explicating and providing an unbiased assessment of the overall effectiveness of FTDCs. Contingent on the extant evidence, this review may also enable examination of the specific components or conditions that moderate the effectiveness of FTDCs for parental legal and psychosocial outcomes. As such, this review will provide practitioners and policy makers with a comprehensive and robust summary of the extant evaluation evidence for FTDCs to inform their future decision-making.

3.1 Criteria for including and excluding studies

3.2 Search strategy

During a piloting phase, it was found that the search below provided the optimum balance between sensitivity and specificity. Due to the small number of search results captured in the pilot and the variation that was found in abstracts across disciplines, the systematic search for the review will utilize a broad search that uses only intervention search terms.

Most existing impact evaluations of FTDCs utilize quasi-experimental designs. For example, Worcel, Furrer, Green, Burrus, and Finigan (2008) used propensity score matching to examine the impact of FTDCs (n = 301 families) compared to business-as-usual child welfare services (n = 1,220 families). Outcomes included reunification, whether participants entered treatment sooner, whether participants spend longer in treatment, and treatment completion. Similar impact evaluations are reported by Bruns et al. (2012) and Lloyd (2015).

3.4 Criteria for determination of independent findings

The software that will be used for this review enables nesting of multiple dependent documents pertaining to one study. Should there be dependent studies, all studies will be coded and data will be extracted from the most complete report of the study and the study will only be included once in a meta-analysis for each conceptually unique outcome. If studies report on multiple conceptually similar outcomes, the effect sizes will be averaged using the method described by Borenstein et al. (2009). If a study utilizes a research design with clustering (e.g., study sites assigned to conditions), the method suggested by Fu et al. (2013) and Higgins and Green (2011) will be used to adjust the standard error (SE). In these cases, where studies do not report the required intra-class correlation coefficient (ICC), the approach taken by Barlow, Bergman, Kornør, Wei, and Bennett (2016) will implemented to assess the impact of clustering on effect estimates. Specifically, Barlow et al.'s (2016) systematic review of group-based parenting interventions took the approach of conducting sensitivity analyses to examine whether the results of their meta-analyses varied with ICCs of 0, 0.03, 0.02 and 0.1.

3.5 Details of study coding categories

3.5.1 Title and abstract screening

The initial phase of assessing study eligibility will begin with title and abstract screening all unique records identified by the systematic search. After removing duplicates and ineligible document types (e.g., book reviews, blog posts) from the results of the systematic search, all records will be imported into the review management software, SysReview (Higginson & Neville, 2014) Each title and abstract (record) will then be assessed according to the following exclusion criteria

(1)

Ineligible document type

(2)

Document is not unique

(3)

Document is not about Family Treatment Drug Courts.

Although all efforts will be made to remove ineligible document types and duplicates prior to screening, automated and manual cleaning can be less than perfect. As such, the first two exclusion criteria will be used to remove ineligible document types and duplicates prior to screening each record on substantive content relevance.

Records retained at the title and abstract screening stage will progress to literature retrieval, where the full-text document will be located and attached within SysReview before progressing to full-text eligibility screening. Where full-text documents cannot be retrieved via existing university resources, they will be ordered through the university libraries of the review authors or by contacting study authors.

3.5.2 Full-text eligibility screening

The full-text of each document progressing through the literature retrieval stage will be screened for final eligibility according to the following exclusion criteria

(1)

Ineligible document type

(2)

Document is not unique

(3)

Document is not an FTDC study

(4)

Ineligible participants

(5)

Ineligible outcome measure

(6)

Ineligible research design.

Although all efforts will be made to remove ineligible document types and duplicates in prior stages, these types of records can sometimes progress into later stages, for example, where duplicate records are not adjacent to each other during screening or where screeners cannot unequivocally determine if record is ineligible based on the title and abstract. As such, the first two exclusion criteria will be used to remove ineligible document types and duplicates prior to screening each document for final eligibility.

3.5.3 Full-text coding and risk of bias assessment

Eligible studies progressing from the full-text screening stage will be coded within SysReview, using the coding companion provided in Appendix C. Broadly, studies will be coded according to the following domains:

(1)

General study characteristics (e.g., document type, study location)

(2)

Participants (e.g., sample characteristics by condition)

(3)

Intervention (e.g., intervention components, intensity, setting)

(4)

Outcomes (e.g., conceptualization, mode of measurement, time-points)

(5)

Research methodology (e.g., design, unit and type of assignment)

(6)

Effect size data

(7)

Risk of bias.

Risk of bias will be evaluated using either the Cochrane randomized or nonrandomized risk of bias tools, whereby studies will be rated across domains as having high, low or unclear risk of bias. Where a domain is rated as “unclear” study authors will be contacted to obtain missing data. Results of the risk of bias assessment will be presented in summary tables and in a risk of bias summary figure. Depending on the data available, sensitivity analysis will be used to examine the impact of risk of bias on effect estimates and corresponding confidence intervals. Possible analyses include: forest plots stratified by level of risk, moderator analysis, or meta-regression. The degree of variation in risk of bias across included studies will determine the approach taken to incorporate risk of bias in statistical analyses. For example, statistical analysis may be stratified by level of risk or all studies may be included in one analysis with a narrative discussion of the risk of bias (see Higgins & Green, 2011, for more detail).

3.6 Statistical procedures and conventions

Statistical analyses will utilize the random effects inverse variance method (Lipsey & Wilson, 2001) and will be performed in R using the rmeta program code available at https://CRAN.R-project.org/package=rmeta (Lumley, 2015). FTDC evaluations typically report binary outcomes (e.g., parental substance abuse relapse: yes/no), and in these cases, effect sizes will be computed as odds ratios. Where outcomes are reported as continuous measures, Hedges g (standardized mean differences, SMDs) will be computed and then transformed into odds ratios for meta-analyses (see, Borenstein, Hedges, Higgins, & Rothstein, 2009). Mean effect sizes will be reported along with their corresponding confidence intervals, both in-text and in forest plots.

Where studies report multiple points of follow-up, effect sizes will be calculated for each time-point, but synthesized separately with studies that have similar outcome time-points. If component studies report baseline and postintervention outcome data, SMDs will be calculated using baseline adjusted mean differences (i.e., mean change scores) and the change score standard deviations, will be standardized using the raw standard deviation within groups. Where authors do not report the standard deviation for mean change scores, Lipsey and Wilson's (2001) formula will be used to calculate the standard deviation (

s T 1 + 2 s T 2 2 / 2 2 ( 1 − r )

). If studies report follow-up outcome data, post-only outcome data will be used to estimate SMDs, and follow-up outcomes will be analysed separately from postintervention outcomes.

Heterogeneity of the studies will be examined using the I ² statistic, χ ² test and τ ² (Higgins & Thompson, 2002). Using the variables outlined in the Objectives section, moderator analyses will be used to explore potential sources of heterogeneity. Specifically, the analogue to analysis of variance will be used for categorical moderators and regression-based approaches will be used for continuous moderators. Depending on the data reported in included studies, additional exploratory subgroup analyses may be performed, however, we will clearly distinguish between a priori and exploratory analyses in our reporting.

Assessment of publication bias will be the final stage of analysis and will first entail inspection of funnel plots for asymmetry to identify whether effect size estimates are influenced by publication bias. If asymmetry is detected, subgroup analyses will be conducted to assess whether effect sizes significantly differ by publication status of the included studies.

ROLES AND RESPONSIBILITIES

Suzanna Fay-Ramirez is a comparative criminologist with extensive experience in researching juvenile, adult and family court models (Fay-Ramirez, 2015) as well as child maltreatment and child welfare responses more broadly. Her research focuses on examining the effectiveness of implementing court programs, such as drug courts, in the United States and Australian context. She has extensive experience and training in high-level statistical methods for the social sciences, as well as qualitative methods of inquiry, and is a frequent reviewer for systematic review studies and experimental methods.

Elizabeth Eggins has co-authored and managed a range of research projects grounded in systematic review methodology, including Campbell Collaboration and industry funded systematic reviews, and scoping or qualitative research that uses systematic search, screening, and coding techniques. Her research focuses on vulnerable families more generally, with a particular focus on quantitative impact evaluations and systematic reviews.

Content: Fay-Ramirez, Eggins

SOURCES OF SUPPORT

Elizabeth's contribution to this review has been supported by an Australian Government Research Training Program Scholarship.

DECLARATIONS OF INTEREST

Suzanna Fay-Ramirez has published in the area of FTDCs, yet has no investment (financial or otherwise) in the results of individual evaluation studies or the results this review.

PRELIMINARY TIMEFRAME

We will submit a draft of the final review in January 2019.

PLANS FOR UPDATING THE REVIEW

Both authors will update this review 3 years after the publication of the first final review.

AUTHOR DECLARATION