PROTOCOL: Multisystemic Therapy for social,emotional,and behavioral problems in youth aged 10–17

Replication

There are more than 250 licensed MST teams in North America and Europe, treating approximately 10,000 serious juvenile offenders and other youth with serious social, emotional, and behavioral problems each year ( Henggeler 2003 ). Considerable attention has been paid to the transportability and dissemination of MST, and to the fidelity of MST replications (e.g., Henggeler 2002b , Schoenwald 2000b , Schoenwald 2001 ). ‘Treatment adherence is optimized by quality assurance mechanisms that … include task-oriented on-site supervision, measurement of adherence to the treatment model using research-validated instruments, and intensive training for all MST staff including a five day orientation training, weekly case consultation with an MST expert, and quarterly booster training’ ( MST Services 2003 ).

Research

Funding for research on MST rose from $5 million (US dollars) in 1995 to approximately $18 million in 2000 to $35 million in 2003 ( Henggeler 2003 ). At least 15 randomised controlled trials (RCTs) have been conducted to assess the impacts of MST, of which the results of eight are published ( Borduin 1990 , Borduin 1995 , Henggeler 1992 , Henggeler 1997 , Henggeler 1999a , Henggeler 1999b , Leschied 2002 , Ogden [in press]). Approximately 30 further RCTS are underway ( MST Services 2003 ). Most of these studies have been or are being conducted by the developers of MST, based at the Family Services Research Center at the Medical University of South Carolina, USA.

Previous reviews

MST trials have been included in meta-analytic reviews of effects of a wider array of interventions with juvenile offenders ( Lipsey 1998 ), family treatment of youth delinquency ( Latimer 2001 ), and family and parenting interventions for conduct disorder and delinquency ( Woolfenden 2002 , Woolfenden 2003 ). However, there are no systematic reviews of the impacts of MST per se.

Results of MST outcome studies have been summarized in non-systematic reviews of the effects of family preservation services ( Fraser 1997b ), interventions for child physical and sexual abuse ( Swenson 2003 ), treatment for substance abuse ( NIDA 1999 ), treatment for delinquency and disruptive behavior in youth ( Smith 1997 ) children's mental health services ( Burns 2004 , Burns 2000 , Kazdin 1998 ), and programs to reduce crime ( Aos 2001 , US DHHS 2001 ). Several reviewers suggested that MST is one of the most promising empirically-based treatments for children and youth ( Hoagwood 2001 , Kazdin 1998 ). ‘MST is unique insofar as providing multiple replications across problems, therapists, and settings. This shows that the treatment and methods of decision making can be extended and that treatment effects are reliable’ ( Kazdin 1998 , pp. 27–28). Chorpita and colleagues classified MST as a ‘probably efficacious treatment’ for conduct and oppositional disorders, but noted that ‘no studies to date support MST other than those conducted by its developers’ ( Chorpita 2002 , p. 177).

Using data from three studies of effects of MST on criminal outcomes, Aos and colleagues ( Aos 2001 ) reported a mean effect size of −.31 (sd=.10). They estimated that the net direct cost of the program per participant was $4,743 (US dollars). When they compared this cost with estimated economic benefits of anticipated reductions in crime, the estimated net benefits of MST range from $31,661 (for taxpayers only) to $131,918 (for taxpayers and crime victims) per MST program participant. Thus, a program that served ten participant families would be expected to produce a net savings of $316,610 in public funds plus over $1 million in savings to potential crime victims.

Types of studies

The review will be limited to experimental studies in which participants were randomly assigned to groups. Studies using other (quasi-experimental) group designs will be identified, but will not be included in this review.

Types of participants

Children and youth (ages 10–17) with severe social, emotional, and behavioral problems, and their family members. These youth may be at risk of out-of-home placement. Participants include:

abused, neglected, and dependent children and youth who may be at risk of foster care or other out-of-home placements in child welfare settings;

Types of interventions

Multisystemic treatment (as defined above) will be compared with ‘usual services,’ other treatment conditions, and no treatment.

Types of outcome measures

Measures of behavioral, psychosocial, and family outcomes will be examined.

Behavioral outcomes include antisocial behavior (as measured by arrest, conviction of a criminal offense), drug use (self-reports and biologic measures), and school attendance.

Selection of trials

Two reviewers (JL and BF) will independently screen titles and abstracts identified in the search and indicate which reports should be retrieved. If there is not enough information in the title and abstract to make such decisions, the full text will be retrieved. Both reviewers will independently read full reports and determine whether these studies meet the inclusion criteria. Selection decisions will be reviewed and any disagreements will be resolved by the review team. Specific reasons for exclusion will be documented for each study that does not meet inclusion criteria.

Assessment of methodological quality

Assessment of methodological quality

Random allocation is an inclusion criterion for this review, given that research indicating its importance in minimising bias ( Schulz 1995 ). Two reviewers (JL & BF) will independently assign each selected study to quality categories described in the Cochrane Handbook ( Clarke 2004 ) where:

indicates adequate concealment of the allocation (for example, by telephone randomisation, or use of consecutively numbered, sealed, opaque envelopes);

For the purposes of this review, only trials meeting categories (A) and (B) will be included. Included studies will also be assessed on: adequate implementation of random assignment, standardization and blinding of assessments, attrition, and intent-to-treat analysis. Studies will be rank-ordered in terms of their ability to support intent-to-treat analysis and use of standardized or objective outcome measures.

Data management

Information on study design and implementation, sample characteristics, intervention characteristics, and outcomes will be extracted from studies and coded on a data extraction form. Two reviewers (JL and MP) will independently code all studies. Differences between coders will be resolved in order to refine coding schemes and establish inter-rater reliability. Citations and data will be entered and organized in RevMan 4.2.3. Authors of studies with missing data will be contacted.

Data synthesis and analysis

Data synthesis will be conducted with RevMan 4.2.3, the latest version of the Cochrane Collaboration's meta-analysis software.

Continuous data will be analyzed if means and standard deviations were available and there is no clear evidence of skew in the distribution. Where scales measured the same clinical outcomes (e.g., psychiatric symptoms) in different ways, standardized mean differences (SMD) will be compared across studies. The RevMan formula for SMD is Hedge's g, which is like Cohen's d but includes an adjustment for small sample bias. Inverse variance methods will be used to pool SMDs, so that each effect size is weighted by the inverse of its variance in an overall estimate of effect size. Confidence intervals of 95% will be used for individual study data and pooled estimates.

Binary outcomes will be analyzed by calculating odds ratios with 95% confidence intervals. Although the odds ratio provides an effect for use in meta-analysis ( Lipsey 2001 ), attempts will be made to preserve information about base rates (actual proportions) and differences in proportions, since this information is of interest to policy makers. RevMan 4.2 uses Mantel-Haenszel methods for combining binary outcome data across studies.

If some primary studies report an outcome (e.g., recidivism) as a dichotomous measure and others use a continuous measure of the same construct, two separate mea-analysis will be used (one for odds ratios and another for SMDs). When a primary outcome study provides multiple measures of the same construct (e.g., parent and youth reports on family cohesion) at the same point in time, an average effect size will be used to avoid dependence problems. When a primary outcome study reports multiple measures of the same construct at different points in time, we will use a single measure that is closest to a one-year follow-up.

Both fixed effect and random effects models will be examined. Heterogeneity will be evaluated with I², the Chi-square test of heterogeneity text, and by comparing results of fixed and random effects models ( Higgins 2002 ). If there is evidence of heterogeneity, possible sources will be investigated.

Assessing bias

Funnel plots will be drawn to investigate relationships between effect size and study precision. Such a relationship could be due to publication or related biases or to systematic differences between small and large studies. If a relationship is identified, the clinical diversity of the studies will be examined as a possible explanation.

Subgroup analysis

Subgroup analyses will focus on service settings (juvenile justice, mental health, and child welfare), type of presenting problem, and the studies' independence from program developers. Depending on the number of primary studies in the analysis, attempts will be made to study variations in effect sizes between studies, using weighted multiple regression or categorical comparisons. These analyses will be exploratory, because they involve non-experimental (cross-study) comparisons. Moderator variables include service setting, presenting problem, duration of the observation period, counterfactual condition (i.e., ‘usual services’, other treatment conditions, no treatment), and independence from program developers.

Sensitivity analysis

Sensitivity analysis will be used to examine the robustness of conclusions. Separate analyses will examine studies that support intent-to-treat analysis and those in which the outcome assessment was blind to treatment allocation.

Use of qualitative research

The narrative review will draw on available qualitative data within studies to discuss program processes and implementation issues.

Use of data on program costs

We will summarize available data on the costs of programs within the studies under review.

Implications for practice

Implications for research