Within-Group Effect-Size Benchmarks for Trauma-Focused Cognitive Behavioral Therapy With Children and Adolescents

Abstract

Purpose:

This article provides benchmark data on within-group effect sizes from published randomized clinical trials (RCTs) supporting the efficacy of trauma-focused cognitive behavioral therapy (TF-CBT) for traumatized children.

Methods:

Within-group effect-size benchmarks for symptoms of trauma, anxiety, and depression were calculated via the Glass approach and adjusted for sample size using Hedges’ g.

Results:

Overall TF-CBT and control group benchmarks are presented, as well as specific benchmarks for sexual abuse and mixed trauma, and whether included studies utilized intent-to-treat analysis.

Discussion:

Community practitioners can use these benchmarks as a comparison tool to evaluate whether the way they are adopting or adapting the TF-CBT intervention is satisfactory, needs to be modified, or should be replaced by a different intervention approach. These benchmarks also have potential utility for future implementation research on TF-CBT assessing which service provision conditions are associated with effect sizes approximating benchmarks provided in this article.

Keywords

TF-CBT benchmarking child abuse sexual abuse research supported interventions implementation science

The process of providing strong research support for the efficacy of various human service interventions has been followed by recognition of the need to study and improve the implementation of these interventions in community-based practice settings (Brownson, Colditz, & Proctor, 2012; Glisson & Schoenwald, 2005; Hoagwood, Burns, Kiser, Ringeisen, & Schoenwald, 2001). Despite the accumulation of rigorous randomized clinical trials (RCTs) and meta-analyses providing empirical support for the efficacy of a number of these interventions, disappointing outcomes have been observed when they are implemented in everyday practice settings (Embry & Biglan, 2008; Weisz, Ugueto, Cheron, & Herren, 2013). Consequently, community-based practitioners should not merely assume that an intervention is appropriate for their clients or practice setting or simply because it has strong research support. Instead, practitioners should monitor pre- to postclient outcomes as one basis for deciding whether the intervention is a good “fit.”

Implementation scientists have postulated various explanations about why these differences in outcomes exist and offer a variety of remedies to reduce this research-to-practice gap (Brownson et al., 2012). Some focus on problems on the fidelity with which community-based practice settings are implementing the interventions. Accordingly, improving community-based outcomes would require improving intervention fidelity and require closer adherence to treatment manuals (D. J. Cohen et al., 2008). A contrasting approach emphasizes not only issues with fidelity but also issues often faced in nonresearch settings trying to implement research supported interventions such as (1) limited resources for staffing, supervision, training, and evaluation of outcomes; (2) the differences in clientele needs and preferences regarding intervention approaches and how they are delivered; (3) higher staff turnover and high client attendance concerns; and (4) workers with larger and more diverse caseloads including more ethnic minority clients (Briere & Scott, 2014; Drake et al., 2001; Rubin, Parrish, & Washburn, 2014). This approach acknowledges potential external validity limitations of RCTs and emphasizes the need for satisfactory ways to modify and adapt intervention(s) to the practice setting and its clients rather than try to achieve maximum treatment fidelity.

In light of these differences, some have even recommended eschewing efforts to implement any particular research supported intervention in community settings and instead focus on a common elements approach (Barth et al., 2012; Chorpita, Becker, Daleiden, & Hamilton, 2007). The common elements approach involves identifying a broad array of components that are shared by various research supported interventions. For example, when working with adult trauma survivors, Briere and Scott (2014) propose that an empathic therapeutic relationship, psychoeducation, reducing stress, providing affect regulation training, developing a trauma narrative, memory processing, and increasing self-awareness and self-acceptance all contribute to positive client outcomes. For the optimal treatment of traumatized children, the addition of a parental education and involvement component is recommended (J. A. Cohen, Mannarino, & Deblinger, 2006).

Regardless of which of the above approaches one might prefer for reducing the gap between RCT outcomes and community-based outcomes, community providers should thoughtfully and consistently assess the outcomes of their efforts to provide research supported interventions. An accumulation of these assessments can be analyzed to identify the service provision characteristics that are associated with better and worse community-based outcomes. Developing this knowledge base requires surmounting a daunting obstacle: the difficulty of conducting outcome studies with control groups in nonresearch practice settings. Fortunately, the inclusion of control groups in these settings—albeit desirable—is not necessary if the aim is not to provide conclusive causal inferences about the effectiveness of an intervention but instead to identify the service provision conditions associated with the best outcomes for clients receiving that intervention.

To address the problem of limited availability of control groups in community-based intervention settings, a benchmarking strategy is gaining attention in clinical psychology (Minami, Wampold, Serlin, Kircher, & Brown, 2007; Spilka & Dobson, 2015) and social work (Rubin, 2014; Rubin et al., 2014; Rubin & Yu, 2015). This strategy involves calculating within-group effect sizes for clients treated in settings where control groups are not feasible, and comparing them to within-group effect-size benchmarks which have been calculated from synthesizing existing RCT data.

It bears emphasizing that the point of the benchmarking approach is not to evaluate the efficacy of an intervention that already has strong research support. The community practice setting’s within-group effect-size would not purport to establish causality; instead, it would be a descriptive datum that can be compared to benchmarks from the RCTs to offer an empirical basis for decisions about continuing, modifying, or replacing the intervention being used. Providers in community-based practice settings commonly make such decisions without obtaining their own control group results. Thus, benchmark comparisons would improve the empirical basis for making such decisions.

The benchmarking approach offers a strategy to help bridge the research–practice gap. One way to achieve this is to correlate within-group effect sizes to variations in service provision across practice settings that share the same target focus. For example, the effect sizes from agencies providing trauma-focused cognitive behavioral therapy (TF-CBT) to children can be examined in relation to the variations in the way the intervention is provided or in relation to the characteristics of the children receiving the intervention. Another advantage of this benchmarking strategy is that practitioners and administrators in the community-based practice settings generate data that can be compared to within-group benchmarks from the RCTs as a basis for deciding whether the way they are adopting or adapting this intervention is satisfactory. It is also a way to help them to improve client outcomes, through exploring whether the intervention should be modified or possibly replaced by a different intervention approach. Evidence that the intervention approach that they are taking is satisfactory would result from within-group effect sizes much closer to the benchmarked aggregate within-group effect sizes of the TF-CBT treatment groups in the RCTs than to those of the control groups. Conversely, if their within-group effect size more closely approximated the control group benchmark, that evidence would imply the need to consider modifying or replacing their current treatment approach.

A recent study by Rubin, Parrish, and Washburn (2014) provided benchmarks regarding research-supported treatments for traumatic stress among adults. The current study provides benchmarks for an intervention widely recognized for its effectiveness in treating traumatized children and youth: TF-CBT. By providing these within-group effect-size benchmarks, the current study seeks to enhance the empirical basis for decisions by community providers of TF-CBT regarding whether they should continue to deliver it in the same way, modify it, or replace it with another intervention in the hopes of continually improving client outcomes.

TF-CBT

TF-CBT is a conjoint parent/child intervention for traumatized youth. Its treatment components emphasize elements of exposure therapy and cognitive processing therapy but also include parenting skills, relaxation training, affect expression and regulation, development of adaptive coping skills, and safety planning. Although TF-CBT has been used to treat various types of childhood traumas, its primary focus has been on the treatment of childhood sexual abuse. For a fuller description of TF-CBT treatment, readers are referred to J. A. Cohen, Mannarino, and Deblinger (2006).

A recent Campbell Collaboration systematic review by Macdonald et al. (2012) supported the potential efficacy of a variety of behavioral or cognitive behavioral interventions in treating the adverse consequences of child sexual abuse. Some of the studies that they reviewed involved the conjoint parent–child intervention now commonly referred to with the TF-CBT acronym (J. A. Cohen et al., 2006). Others included children, only, as recipients of therapy and may have been described as behavioral interventions that included the use of cognitive behavioral techniques. Cary and McMillen (2012) note that there are “branded” and “nonbranded” versions of TF-CBT. The branded version has been manualized, widely disseminated, promoted, and funded through the National Child Traumatic Stress Network and provided via online training (Medical University of South Carolina, 2005). Studies of nonbranded versions might not use the TF-CBT label, instead referring to the treatment with terms like cognitive behavioral therapy for children or youth; however, the intervention components are quite similar to those of the branded version Cary and McMillen (2012) also refer to the nonbranded versions as “prebranded” because the studies on them often predated the studies on the branded version and found similar outcomes for both the branded and the prebranded versions of TF-CBT in their review. In light of the similarities in method and results, we did not exclude nonbranded or prebranded versions from our study unless they did not include a conjoint parent/child component. Additional comprehensive reviews of the literature supporting the efficacy of TF-CBT have been provided (Gillies, Taylor, Gray, O’Brien, & D’Abrew, 2013; Harvey & Taylor, 2010; Kliem & Kroger, 2013; Kowalik, Weller, Venter, & Drachman, 2011). These reviews and others related to the trauma treatment of children are listed in Appendix A.

Method

Study Search and Selection

An initial broad Internet and EBSCOhost database search was conducted in the spring of 2014 for meta-analyses and systemic reviews published since 1990 on the efficacy of TF-CBT and other cognitive behavioral interventions provided to children and youth. This included the following databases: Google Scholar, Web of Science, Medline, Academic Search Complete, ERIC, Social Work Abstracts, Soc INDEX with full text, Social Sciences full text, PubMed, PsychARTICLES and PsycINFO, Psychology and Behavioral Sciences Collection, Health and Psychosocial Instruments, Health Source Nursing/Academic Edition, Families Studies Abstracts, and the Cochrane and Campbell libraries. Search terms included “child (youth, adolescent, teen) trauma, childhood trauma, child sexual abuse, trauma treatment, child trauma treatment, cognitive behavioral treatment of trauma, trauma-focused cognitive behavioral therapy, child cognitive behavioral therapy” and combinations of these terms with the addition of “systematic review” and “meta-analysis.” Seven meta-analyses and five systematic reviews were found. An initial screening of the title of each RCT article regarding TF-CBT and youth trauma treatment identified in those meta-analyses and systematic reviews was conducted. It yielded 57 studies. Next, a targeted search was executed in fall 2014 using the same databases to locate additional RCTs on the efficacy of TF-CBT that were published too late to be included in the meta-analyses and reviews. It yielded five additional studies.

Our next level of review defines our inclusion criteria including the use of an RCT design or a design that had a TF-CBT group only; at least one standardized outcome measure of trauma symptomatology, anxiety, or depression; and the provision of either a branded or a prebranded form of TF-CBT that mirrored the essential elements of branded TF-CBT including a conjoint/parent–child component. There was significant overlap of exclusion criteria within studies reviewed; therefore, reasons for exclusion are not mutually exclusive. The primary reason for “exclusion” was based on the authors’ judgment, and eliminated articles potentially could have met multiple exclusion criteria.

Figure 1 shows the progression of the article selection process. Of the 57 studies initially identified, 23 were eliminated because they used eye movement desensitization and reprocessing or another intervention that was not TF-CBT. Five studies were excluded because they reported measures that were not directly related to anxiety, trauma, or depression. Five studies were excluded because they did not focus exclusively on treating children. One study was excluded due to outcome measures focusing exclusively on the parent, not the child, and two studies were excluded because they did not include a parental involvement component. Two studies were excluded because the intervention was provided by a lay provider rather than a master- or doctoral-level mental health clinician. Three more were eliminated because they were not provided in an outpatient setting. One study was excluded because the child’s treatment for childhood sexual abuse was conducted with the offending parent. Three additional studies were excluded because they were follow-up studies reiterating the pre–post data that were reported in other studies in our sample. Finally, one additional study was eliminated because it compared different versions of TF-CBT to one another.

The rationale for including studies with only a TF-CBT treatment group was that the absence of a control group obviated the problem of a selectivity bias affecting group assignment, and because our study focuses on within-group effect sizes only. Because our proposed benchmark calculations were in the form of standardized within-group effect sizes in which the difference between pretest and posttest means is divided by the pretest standard deviation—done separately for the experimental and control groups—another inclusion criterion required that the article report those statistics separately for each of those groups. These selection parameters resulted in a final sample of 17 studies, all of which had identified the training and experience of the interventionists and also had identified how treatment fidelity was monitored. Five of these studies did not have a waitlist or treatment as usual (TAU) control group, instead comparing TF-CBT to another empirically supported treatment or to TF-CBT with the addition of medication. For these studies, we included only the data for the TF-CBT groups alone, which were used in our calculations of TF-CBT within-group benchmarks.

Data Collection Process

The following information was recorded for each study: authors and year of study, sample size of each group, whether assessors were blind to treatment condition, type of control condition (waitlist, supportive therapy, or TAU), target population (% Caucasian/White and % Female), whether an intent-to-treat analysis was conducted, percentage attrition in control and treatment groups, and type of trauma reported. Outcome data were recorded in five separate categories. One category included commonly used standardized interviewer administered measures of level of trauma symptomatology such as the Kiddie Schedule for Affective Disorders and Schizophrenia (Kaufman et al., 1997) or the Clinician-Administered Post-Traumatic Stress Disorder (PTSD) Scale for Children and Adolescents (Newman et al., 2004). Three other categories included standardized self-report measures of the severity of trauma symptoms, depression, or anxiety. These measures included the Child PTSD Symptom Scale (Foa, Johnson, Feeny, & Treadwell, 2001), the Children’s Depression Inventory (Kovacs, 1984), and the State-Trait Anxiety Inventory for Children (Spielberger, 1973). A fifth category included the Child Behavior Checklist (CBCL), which identifies a range of emotional and behavioral problems in children, including the target constructs (Achenbach et al., 2003). This measure was the one most frequently used across studies and was included in 65% of the articles meeting all of the selection criteria. Therefore, we decided to establish a benchmark for this instrument alone due to its widespread use in clinical practice of trauma treatment.

Follow-up (after posttest) data were not included because providers in settings for whom our benchmarks are provided rarely conduct follow-up assessments of clients months after treatment completion. Two authors independently judged whether each article met the inclusion criteria and independently recorded data from the included articles. There was initial disagreement about inclusion for 3 of the 57 originally identified articles. The few discrepancies were resolved via a joint reexamination of the original article by the first and second author. There was a 99% interrater agreement rate for data recorded from each of the included articles. A listing of studies included in our benchmark calculations can be found in Appendix B.

Benchmark Calculations

Glass’s delta approach (Glass, 1976) was used to calculate within-group effect sizes. This involved dividing the difference between the pretest and posttest means by the pretest standard deviation. Separate calculations were performed for the experimental group and for the control group so as to get separate within-group effect sizes for recipients of TF-CBT and for the controls (Feingold, 2009; Kadel & Kip, 2012). Hedges’ g was then calculated to adjust for small sample sizes of some of the studies. Hedges’ g is calculated by a formula in which the effect size is multiplied by 1 minus (3 over [4N − 9]; Wilson, 2011). This calculation yields a more conservative estimate of effect size. The individual study within-group effect sizes were then averaged across studies using a two-step calculation process (Minami et al., 2007). First, the variance of the individual study within-group effect sizes was estimated as follows: $2 (1 - r_{i}) / n_{i} + g_{i}^{2} / 2 n_{i}$ . Next, the variance and effect size (g_i ) were used to estimate the fixed benchmark effect size across studies using the following formula: $g_{B} = [Σ_{i} g_{i} / δ_{g (i)}^{2}] over [Σ_{i} 1 / δ_{g (i)}^{2}]$ .

Results

Table 1 displays the aggregate within-group effect sizes (g _Hedges) for the TF-CBT recipients and for the controls by type of outcome measure used. For example, the aggregate within-group effect size measured when outcome was measured by a trauma symptom interview was 1.48 for TF-CBT recipients and 0.82 for controls. The effect sizes were lower for the self-report measures. According to this table, if a within-group effect size (Hedges’ g) based on pre- to posttrauma symptom interviews of TF-CBT recipients in a practice setting were to come much closer to 1.48 than to 0.82, it would provide grounds for optimism about the way the provider was providing TF-CBT, in that its clients were achieving outcomes much better than the average outcome for the RCT controls, and approximating the TF-CBT recipients’ average outcome. The same reasoning would apply if the practice setting TF-CBT recipients’ effect sizes on a self-report measure were approximately 0.50 or higher.

Table 1.

Within Group Aggregate Effect Size by Treatment Condition and Outcome Measures.

Intervention Type	Interview	Self-Report Scales
Intervention Type	Trauma Symptoms	Trauma Symptoms	CBCL	Depression	Anxiety
TF-CBT
k	9	8	11	9	9
g _Hedges	1.48	.76	.50	.67	.67
SE_g	0.08	.08	.05	.06	.06
g _Hedges 95% CI	[1.63, 1.32]	[0.91, 0.61]	[0.60, 0.40]	[0.80, 0.56]	[0.80, 0.56]
Controls
k	7	4	7	5	5
g _Hedges	0.82	.67	.34	.36	.37
SE_g	0.07	.08	.06	.06	.06
g _Hedges 95% CI	[0.96, 0.69]	[0.82, 0.51]	[0.47, 0.22]	[0.48, 0.24]	[0.49, 0.25]

Note. The boldface values indicate within-group effect sizes (not between-group effect sizes) benchmark calculations. k = number of studies; g _Hedges = aggregated effect size; SE_g = standard error of aggregate effect size; CI = confidence interval; TF-CBT = trauma-focused cognitive behavioral therapy; CBCL = Child Behavior Checklist.

Similar comparisons could be made to the data in Table 2, which further breaks down the aggregated within-group effect sizes by whether an intent-to-treat analysis was conducted. Smaller aggregate effect sizes were obtained in the intent-to-treat analyses, as these calculations included data from all participants who had been randomized to a treatment group rather than just reporting data for participants completing the intervention. This controls for potentially inflated effect-size calculations.

Table 2.

Within-Group Aggregate Effect-Size by Treatment Condition and Whether an Intent-to-Treat (ITT) Analysis Was Conducted.

	Interview	Self-Report Scales
	Trauma Symptoms	CBCL	Depression	Anxiety
TF-CBT ITT
k	4	3	5	5
g _Hedges	1.41	.32	.70	.76
SE_g	0.11	.10	.08	.09
g _Hedges 95% CI	[1.62, 1.20]	[0.50, 0.13]	[0.86, 0.55]	[0.93, 0.60]
TF-CBT Non-ITT
k	3	8	4	4
g _Hedges	1.57	.57	.65	.80
SE_g	0.12	.06	.09	.09
g _Hedges 95% CI	[1.81, 1.36]	[0.69, 0.45]	[0.82, 0.47]	[0.98, 0.61]
Controls ITT
k	4	3	5	5
g _Hedges	0.77	.24	.41	.30
SE_g	0.09	.10	.08	.07
g _Hedges 95% CI	[0.94, 0.60]	[0.43, 0.06]	[0.55, 0.26]	[0.44, 0.16]
Controls Non-ITT
k	4	5	3	3
g _Hedges	0.96	.83	.31	.49
SE_g	0.11	.11	.09	.10
g _Hedges 95% CI	[1.68, 0.75]	[1.05, 0.61]	[0.49, 0.12]	[0.68, 0.31]

Table 3 breaks down the TF-CBT recipient data by whether participants experienced sexual or nonsexual trauma. The interview benchmarks are similar for both sexual and nonsexual types of trauma. To approximate the aggregate within-group effect sizes of TF-CBT recipients in research studies, the interview-based within-group effect sizes of TF-CBT recipients in practice settings would have to be near 1.6 for sexually abused clients and 1.41 if the caseload was not limited to sexual abuse. For self-reported outcome data, the benchmarks range a bit more widely and were calculated to be between 0.47 and 0.65 for sexual trauma and between 0.85 and 1.20 for mixed trauma.

Table 3.

Within-Group Aggregate Effect-Size for TF-CBT by Type of Trauma.

Type of Trauma	Interview	Self-Report Scales
Type of Trauma	Trauma Symptoms	Trauma Symptoms	Depression	Anxiety
Sexual abuse
k	4	5	5	5
g _Hedges	1.60	.47	.56	.65
SE_g	0.13	.10	.09	.09
g _Hedges 95% CI	[1.85, 1.35]	[0.70, 0.24]	[0.72, 0.41]	[0.81, 0.50]
Mixed trauma
k	5	4	4	4
g _Hedges	1.41	.97	.85	1.20
SE_g	0.10	.10	.10	.11
g _Hedges 95% CI	[1.61, 1.21]	[1.17, 0.78]	[1.04, 0.66]	[1.43, 0.98]

Discussion and Applications to Practice

This study calculated and reported descriptive statistics on aggregate within-group effect sizes from published studies that evaluated the efficacy of TF-CBT in the treatment of traumatized children and youth and a nonoffending caregiver. Community-based practitioners can compare these benchmarks to their own outcomes in providing TF-CBT to children and youth so as to enhance their decisions regarding whether the way they are adopting or adapting this intervention is satisfactory. This comparison can facilitate conversations in an agency concerning potential barriers or challenges to providing TF-CBT in that setting or to the clients they serve such conversations also could consider whether TF-CBT should be modified or potentially replaced by a different intervention approach in an effort to improve client outcomes. Before making these comparisons, community practitioners should appropriately adjust their effect sizes by using the Hedges’ g formula to reflect the number of their TF-CBT recipients.

In addition to comparing their within-group effect size to the aggregate effect sizes for TF-CBT recipients, community practitioners might want to compare the TF-CBT effect sizes for their clients to the control group data in the tables. If their active treatment within-group effect sizes more closely approximate the corresponding TF-CBT effect sizes than the TF-CBT control group effect sizes, the greater the grounds for optimism regarding whether the way they are providing TF-CBT appears to be satisfactory. Otherwise, they might consider modifying the way they are providing TF-CBT or replacing it with a different intervention approach that might be a better fit for their setting or clientele.

The value in calculating the within-group effect size in the practice setting and then comparing it to benchmark data is based on studies indicating that when interventions with strong research support are implemented in everyday practice settings, they tend not be implemented with the same fidelity as in controlled research settings due to differences in training, supervision, staffing, caseload sizes, client attendance issues, and other service provision resources (Chorpita et al., 2007). Moreover, the evidence-based practice (EBP) process encourages practitioners to modify empirically supported treatments if needed to make them a better fit for their clientele characteristics and needs (Rubin & Bellamy, 2012).

Although this study employed some meta-analytic techniques, it is not a full-fledged meta-analysis. Therefore, we did not include unpublished studies because our purpose was to provide benchmarks derived from the studies that provided the research support for the efficacy of TF-CBT. Thus, searching for unpublished literature was outside of the scope of this review as we were not attempting to further establish the efficacy of TF-CBT. Instead, we merely wanted to provide benchmarks to which practitioners providing TF-CBT in everyday practice settings can compare their outcomes. For the same reason, we focused on within-group effect sizes rather than between-group effect sizes that are the foci of meta-analyses. Therefore, J. Cohen’s (1988) standards regarding strong, medium, and weak effects (such as Cohen’s d) do not apply to our benchmarks, because his standards pertain to between-group effect sizes. In that connection, it is worth noting that within-group effect sizes tend to be stronger than between-group effect sizes, since between-group effect sizes are based on the difference in outcome between intervention and control groups, whereas within-group effect sizes are based on pre- to postdifferences only. Thus, if TAU has some impact, even if significantly less than the TF-CBT impact, the difference from pre to post for TF-CBT recipients will be greater than the difference in the degree of pre- to postimprovement between the two groups. Likewise, if factors like contemporaneous events (history) and the passage of time have a beneficial impact on the symptomatology of waitlist controls, it stands to reason that their within-group effect size will show some improvement.

A potential limitation of this study is that our sample consisted of only 17 studies; however, that number exceeds the number of studies (12) that met the inclusion criteria for the Cary and McMillen (2012) systematic review of TF-CBT for use with children and youth. Another potential limitation is that the creators of the TF-CBT protocol, Cohen and Deblinger, were authors on a number of the studies included in the analysis. However, when a comparison was done for experimental group outcomes on the CBCL trauma measure of studies (10) listing them as an author or coauthor versus those that did not have them listed as an author or coauthor (7), the resulting aggregate effect sizes were quite similar and were calculated at 0.48 and 0.54, respectively. Thus, the effect sizes reported in the studies listing Cohen or Deblinger as an author or coauthor do not appear to be inflated or to differ to a meaningful degree from the effect sizes reported in the other studies. Similarly, a risk bias analysis was conducted to explore whether aggregate effect sizes differed based on whether assessors were blinded (11 studies) or not (6 studies), and the differences in these aggregated effect sizes were found to be trivial for both interview and self-report measures.

Some might question the extent to which community-based settings will actually use within-group effect-size benchmarks to inform their service provision decisions. Such skepticism has parallels in doubts about practitioner engagement in the EBP process in general and is understandable in light of studies reporting practitioners’ limited use of empirical data in making practice decisions (Parrish & Rubin, 2012; Washburn & Parrish, 2014). However, just as we would not be able to know whether practitioners use practice-relevant research to inform their practice unless such research existed, the value and utility of the benchmarking strategy cannot be assessed before these benchmarks are provided.

One implication of our study, therefore, is that efforts are needed to help community-based practitioners understand and utilize the benchmarks provided in this and other emerging benchmark studies, such as the studies by Rubin et al. (2014) and Rubin and Yu (2015). For example, those practitioners might need assistance regarding what pre- to postoutcome measures to use and in calculating their within-group effect sizes.

Another implication is the need for studies that evaluate the extent to which community-based practitioners eventually use these benchmarks to inform their practice decisions. Likewise, implementation science studies that describe the various ways that research supported interventions are being adapted in community-based settings—and the service provision characteristics of those settings—can correlate those variations with variations in within-group effect sizes. Such studies might offer implications regarding the conditions under which other community-based settings can feasibly and successfully adapt such interventions. Those studies might also want to compare their within-group effect sizes to the corresponding benchmarks as another way to identify those implementation approaches that appear to be experiencing the most success.

In light of ethical and other considerations as to why it is essential for practitioners to inform their practice decisions with empirical evidence, it is important to develop and test out innovative ways to help practitioners in community-based settings feasibly judge whether the ways they have implemented research supported interventions are adequate or in need of change. One emerging innovative approach provides benchmark data on aggregate within-group effect sizes that are calculated from the pretest and posttest data per group from published studies that have supplied the research support for the efficacy of those interventions. This study has provided such benchmarks derived from the published studies regarding the efficacy of TF-CBT in the treatment of traumatized children and youth.

Footnotes

Appendix A

Table A1.

Summary of Meta-Analyses and Reviews Guiding Literature Selection for Trauma Benchmarks.

Amand, Bard, and Silovsky	2008	Meta-analysis of treatment for child sexual behavior problems: practice elements and outcomes.
Bisson and Andrew	2005	Psychological treatment of post-traumatic stress disorder (PTSD).
Bisson et al.	2007	Psychological treatments for chronic PTSD: systematic review and meta-analysis
Benish, Imel, and Wampold	2008	The relative efficacy of bona fide psychotherapies for treating PTSD: a meta-analysis of direct comparisons.
Cary and McMillen	2012	The data behind the dissemination: a systematic review of trauma-focused cognitive behavioral therapy for use with children and youth.
Dyregov and Yule	2006	A review of PTSD in children.
Gillies, Taylor, Gray, O’Brien, and D’Abrew	2013	Psychological therapies for the treatment of PTSD in children and adolescents.
Harvey and Taylor	2010	A meta-analysis of the effects of psychotherapy with sexually abused children and adolescents.
Kliem and Kroger	2013	Prevention of chronic PTSD with early cognitive behavioral therapy. A meta-analysis using mixed-effects modeling.
Kornør et al.	2008	Early trauma-focused cognitive behavioral therapy to prevent chronic PTSD and related symptoms: a systematic review and meta-analysis.
Kowalik, Weller, Venter, and Drachman	2011	cognitive behavioral therapy for the treatment of pediatric PTSD: a review and meta-analysis.
Macdonald et al.	2012	cognitive behavioral interventions for children who have been sexually abused

Appendix B

Table B1.

Demographics and Effect Sizes for Studies Selected for Inclusion of Benchmark Calculations.

Demographic Characteristics			Clinical Characteristics			Treatment Conditions and Constructs Measured
Group Sample Size	% Total Female	% Total White	Blind Assessor	Trauma Type	ITT	% Dropout	Group	Effect Size Adjusted		SE
Celano, Hazzard, Webb, and McCall (1996)
15	100	22	Y	Sexual	N	40	CBT	Interview	^a	^a
								Trauma	.67	.07
								CBCL	.38	.07
								Anxiety	^a	^a
								Depression	^a	^a
17						29	TAU	Interview	^a	^a
								Trauma	.61	.06
								CBCL	.78	.07
Cohen (1996)
39	58	54	N	Sexual	N	^b	TF-CBT	Interview	^a	^a
								CBCL	.87	.03
								Trauma	^a	^a
								Anxiety	^a	^a
								Depression	^a	^a
28						^b	Supportive	Interview	^a	^a
								CBCL	.26	.04
								Trauma	^a	^a
								Anxiety	^a	^a
								Depression	^a	^a
J. A. Cohen and Mannarino (1998)
30	69	59	N	Sexual	Y	^b	TF-CBT	Interview	^a	^a
								Trauma	^a	^a
								CBCL	.32	.04
								Anxiety	.73	.04
								Depression	.75	.04
30						^b	Supportive	Interview	^a	^a
								Trauma	^a	^a
								CBCL	.07	.05
								Anxiety	.52	.06
								Depression	.06	.05
J. A. Cohen, Deblinger, Mannarino, and Steer (2004)
89	79	60	Y	Sexual	N	^b	TF-CBT	Interview	1.73	.02
								Trauma	^a	^a
								CBCL	.60	.01
								Anxiety	.78	.01
								Depression	.56	.01
91						^b	Supportive	Interview	1.25	.01
								Trauma	^a	^a
								CBCL	.48	.01
								Anxiety	.57	.01
								Depression	.39	.01
J. A. Cohen, Mannarino, and Knudsen (2005)
41	68	60	Y	Sexual	Y	^b	TF-CBT	Interview	^a	^a
								Trauma	.35	.03
								CBCL	.17	.02
								Anxiety	.45	.03
								Depression	.47	.03
41						^b	Supportive	Interview	^a	^a
								Trauma	.16	.02
								CBCL	.03	.02
								Anxiety	.14	.02
								Depression	.03	.02
J. A. Cohen, Mannarino, Perel, and Staron (2007)
11	100	77	Y	Sexual	N	9	TF-CBT	Interview	^a	^a
								CBCL	^a	^a
								Trauma	2.42	.18
								Anxiety	^a	^a
								Depression	^a	^a
J. A. Cohen, Mannarino, and Iyengar (2011)
64	51	56	Y	IPV	Y	33	TF-CBT	Interview	1.28	.02
								Trauma	.40	.02
								CBCL	.34	.02
								Anxiety	.40	.02
								Depression	.29	.02
60						47	Supportive	Interview	.59	.02
								Trauma	.09	.02
								CBCL	.38	.02
								Anxiety	.09	.02
								Depression	.11	.02
Deblinger, McLeer, and Henry (1990)
19	100	^a	N	Sexual	N	^b	TF-CBT	Interview	^a	^a
								CBCL	.53	.06
								Trauma	^a	^a
								Anxiety	^a	^a
								Depression	^a	^a
Deblinger, Lippmann, and Steer (1996)
22	83	72	N	Sexual	N	12	TF-CBT	Interview	2.23	.08
								Trauma	^a	^a
								CBCL	.63	.05
								Anxiety	.67	.05
								Depression	.58	.05
Deblinger, Stauffer, and Steer (2001)
21	61	64	N	Sexual	N	^b	TF-CBT	Interview	.84	.06
								Trauma	^a	^a
								CBCL	.66	.05
								Anxiety	^a	^a
								Depression	^a	^a
21						^b	Supportive	Interview	.62	.05
								Trauma	^a	^a
								CBCL	.41	.05
								Anxiety	^a	^a
								Depression	^a	^a
Diehle et al. (2014)
23	61	^a	Y	Mixed	N	^b	TF-CBT	Interview	1.28	.06
								CBCL	^a	^a
								Trauma	^a	^a
								Anxiety	^a	^a
								Depression	^a	^a
Jensen et al. (2014)
79	80	^a	Y	Mixed	Y	25	TF-CBT	Interview	1.49	.02
								Trauma	1.92	.02
								CBCL	^a	^a
								Anxiety	.90	.01
								Depression	1.77	.02
77						18	TAU	Interview	.88	.02
								Trauma	1.27	.02
								CBCL	^a	^a
								Anxiety	.50	.01
								Depression	.94	.02
King et al. (2000)
12	69	^a	N	Sexual	Y	25	CBT	Interview	4.19	.26
								Trauma	0.0	0.0
								CBCL	.81	.10
								Anxiety	.35	.09
								Depression	.52	.09
12						17	Waitlist	Interview	.54	.10
								Trauma	−.07	.08
								CBCL	.33	.09
								Anxiety	.11	.08
								Depression	.20	.08
de Roos et al. (2011)
18	44	^a	Y	Explosion	Y	23	TF-CBT	Interview	^a	^a
								CBCL	^a	^a
								Trauma	1.30	.08
								Anxiety	.78	.06
								Depression	1.00	.07
Smith et al. (2007)
12	50	46	Y	Mixed	N	^b	TF-CBT	Interview	4.70	.29
								Trauma	2.88	.19
								CBCL	^a	^a
								Anxiety	2.04	.15
								Depression	1.83	.14
12						^b	Waitlist	Interview	.91	.10
								Trauma	.39	.09
								CBCL	^a	^a
								Anxiety	−.04	.08
								Depression	.10	.08
Scheeringa, Weems, Cohen, Amaya-Jackson, & Guthrie (2011)
17	34	35	Y	Mixed	N	58^c	TF-CBT	Interview	1.40	.08
								Trauma	^a	^a
								CBCL	^a	^a
								Anxiety	^a	^a
								Depression	^a	^a
11						54^c	Supportive	Interview	.18	.09
								Trauma	^a	^a
								CBCL	^a	^a
								Anxiety	^a	^a
								Depression	^a	^a
Webb, Hayes, Grasso, Laurenceau, & Deblinger (2014)
72	64	46	Y	Mixed	N	23	TF-CBT	Interview	^a	^a
								CBCL	.53	.01
								Trauma	^a	^a
								Anxiety	^a	^a
								Depression	^a	^a

Note. All measures are self-report unless indicated as a trauma interview measure. Sexual = childhood sexual abuse; Mixed trauma = participants in sample reported diverse trauma experiences; IPV = interpersonal violence; SE = standard error; TF-CBT = trauma-focused cognitive behavioral therapy; CBCL = Child Behavior Checklist; TAU = treatment as usual; CBT = cognitive behavioral therapy.

^aNo data reported. ^bNo data reported/data were not broken down by group. ^cStudy was interrupted by Hurricane Katrina.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Achenbach

T. M.

Rescorla

L. A.

McConaughey

Pecora

Wetherbee

Ruffle

(2003). The Achenbach system of empirically based assessment. Handbook of Psychological and Educational Assessment of Children: Personality, Behavior, and Context, 2, 406–432.

Amand

A. S.

Bard

D. E.

Silovsky

J. F.

(2008). Meta-analysis of treatment for child sexual behavior problems: Practice elements and outcomes. Child Maltreatment, 13, 145–166.

Barth

R. P.

Lee

B. R.

Lindsey

M. A.

Collins

K. S.

Streider

Chorpita

B. F.

… Sparks

J. A.

(2012). Evidence-based practice at a crossroads: The timely emergence of common elements and common factors. Research on Social Work Practice, 22, 108–119.

Benish

S. G.

Imel

Z. E.

Wampold

B. E.

(2008). The relative efficacy of bona fide psychotherapies for treating post-traumatic stress disorder: A meta-analysis of direct comparisons. Clinical Psychology Review, 28, 746–758.

Bisson

Andrew

(2005). Psychological treatment of post-traumatic stress disorder (PTSD). The Cochrane Library. Retrieved from http://www.ncbi.nlm.nih.gov/m/pubmed/15846661/

Bisson

J. I.

Ehlers

Matthews

Pilling

Richards

Turner

(2007). Psychological treatments for chronic post-traumatic stress disorder. The British Journal of Psychiatry, 190, 97–104.

Briere

J. N.

Scott

(2014). Principles of trauma therapy: A guide to symptoms, evaluation and treatment (2nd ed.). New York, NY: Sage.

Brownson

R. C.

Colditz

G. A.

Proctor

E. K.

(2012). Dissemination and implementation research in health: Translating science to practice. New York, NY: Oxford University Press.

Cary

C. E.

McMillen

J. C.

(2012). The data behind the dissemination: A systematic review of trauma-focused cognitive behavioral therapy for use with children and youth. Children and Youth Services Review, 34, 748–757.

10.

Celano

Hazzard

Webb

McCall

(1996). Treatment of traumagenic beliefs among sexually abused girls and their mothers: An evaluation study. Journal of Abnormal Child Psychology, 24, 1–17.

11.

Chorpita

B. F.

Becker

K. D.

Daleiden

E. L.

Hamilton

J. D.

(2007). Understanding the common elements of evidence-based practice: Misconceptions and clinical examples. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 647–652.

12.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York, NY: Lawrence Erlbaum.

13.

Cohen

D. J.

Crabtree

B. F.

Etz

R. S.

Balasubramanian

B. A.

Donahue

K. E.

Leviton

L. C.

… Green

L. W.

(2008). Fidelity versus flexibility: Translating evidence-based research into practice. American Journal of Preventive Medicine, 35, S381–S389.

14.

Cohen

J. A.

Deblinger

Mannarino

A. P.

Steer

R. A.

(2004). A multisite, randomized controlled trial for children with sexual abuse–related PTSD symptoms. Journal of the American Academy of Child & Adolescent Psychiatry, 43, 393–402.

15.

Cohen

J. A.

Mannarino

A. P.

(1996). A treatment outcome study for sexually abused preschool children: Initial findings. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 42–50.

16.

Cohen

J. A.

Mannarino

A. P.

(1998). Interventions for sexually abused children: Initial treatment outcome findings. Child Maltreatment, 3, 17–26.

17.

Cohen

J. A.

Mannarino

A. P.

Deblinger

(2006). Treating trauma and traumatic grief in children and adolescents. New York, NY: Guilford Press.

18.

Cohen

J. A.

Mannarino

A. P.

Iyengar

(2011). Community treatment of posttraumatic stress disorder for children exposed to intimate partner violence: A randomized controlled trial. Archives of Pediatrics & Adolescent Medicine, 165, 16–21.

19.

Cohen

J. A.

Mannarino

A. P.

Knudsen

(2005). Treating sexually abused children: 1 year follow-up of a randomized controlled trial. Child Abuse & Neglect, 29, 135–145.

20.

Cohen

J. A.

Mannarino

A. P.

Perel

J. M.

Staron

(2007). A pilot randomized controlled trial of combined trauma-focused CBT and sertraline for childhood PTSD symptoms. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 811–819.

21.

Deblinger

Lippmann

Steer

(1996). Sexually abused children suffering posttraumatic stress symptoms: Initial treatment outcome findings. Child Maltreatment, 1, 310–321.

22.

Deblinger

McLeer

S. V.

Henry

(1990). Cognitive behavioral treatment for sexually abused children suffering post-traumatic stress: Preliminary findings. Journal of the American Academy of Child & Adolescent Psychiatry, 29, 747–752.

23.

Deblinger

Stauffer

L. B.

Steer

R. A.

(2001). Comparative efficacies of supportive and cognitive behavioral group therapies for young children who have been sexually abused and their non-offending mothers. Child Maltreatment, 6, 332–343.

24.

de Roos

Greenwald

den Hollander-Gijsman

Noorthoorn

van Buuren

De Jongh

(2011). A randomized comparison of cognitive behavioral therapy (CBT) and eye movement desensitization and reprocessing (EMDR) in disaster-exposed children. European Journal of Psychotraumatology, 2, 5694.

25.

Diehle

Opmeer

B. C.

Boer

Mannarino

A. P.

Lindauer

R. J.

(2014). Trauma-focused cognitive behavioral therapy or eye movement desensitization and reprocessing: What works in children with posttraumatic stress symptoms? A randomized controlled trial. European Child & Adolescent Psychiatry, 24, 1–10.

26.

Drake

R. E.

Goldman

H. H.

Leff

H. S.

Lehman

A. F.

Dixon

Mueser

K. T.

Torrey

W. C.

(2001). Implementing evidence-based practices in routine mental health service settings. Psychiatric Services, 52, 179–182.

27.

Embry

D. D.

Biglan

(2008). Evidence-based kernels: Fundamental units of behavioral influence. Clinical Child and Family Psychology Review, 11, 75–113.

28.

Feingold

(2009). Effect Sizes for growth-modeling analysis for controlled clinical trials in the same metric as for classical analysis. Psychological Methods, 14, 43–53.

29.

Foa

E. B.

Johnson

K. M.

Feeny

N. C.

Treadwell

K. R.

(2001). The Child PTSD Symptom Scale: A preliminary examination of its psychometric properties. Journal of Clinical Child Psychology, 30, 376–384.

30.

Gillies

Taylor

Gray

O’Brien

D’Abrew

(2013). Psychological therapies for the treatment of post-traumatic stress disorder in children and adolescents (Review). Evidence-Based Child Health: A Cochrane Review Journal, 8, 1004–1116.

31.

Glass

G. V.

(1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3–8.

32.

Glisson

Schoenwald

S. K.

(2005). The ARC organizational and community intervention strategy for implementing evidence-based children’s mental health treatments. Mental Health Services Research, 7, 243–259.

33.

Harvey

S. T.

Taylor

J. E.

(2010). A meta-analysis of the effects of psychotherapy with sexually abused children and adolescents. Clinical Psychology Review, 30, 517–535.

34.

Hoagwood

Burns

B. J.

Kiser

Ringeisen

Schoenwald

S. K.

(2001). Evidence-based practice in child and adolescent mental health services. Psychiatric Services, 52, 1179–1189.

35.

Jensen

T. K.

Holt

Ormhaug

S. M.

Egeland

Granly

Hoaas

L. C.

… Wentzel-Larsen

(2014). A randomized effectiveness study comparing trauma-focused cognitive behavioral therapy with therapy as usual for youth. Journal of Clinical Child & Adolescent Psychology, 43, 356–369.

36.

Kadel

R. P.

Kip

K. E.

(2012). A SAS macro to compute effect size (Cohen’s d) and its confidence interval from raw survey data. Retrieved from Analytics.ncsu.edu/2012/SD-06.pdf

37.

Kaufman

Birmaher

Brent

Rao

U. M. A.

Flynn

Moreci

… Ryan

(1997). Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL): Initial reliability and validity data. Journal of the American Academy of Child & Adolescent Psychiatry, 36, 980–988.

38.

King

N. J.

Tonge

B. J.

Mullen

Myerson

Heyne

Rollings

… Ollendick

T. H.

(2000). Treating sexually abused children with posttraumatic stress symptoms: A randomized clinical trial. Journal of the American Academy of Child & Adolescent Psychiatry, 39, 1347–1355.

39.

Kliem

Kröger

(2013). Prevention of chronic PTSD with early cognitive behavioral therapy. A meta-analysis using mixed-effects modeling. Behaviour Research and Therapy, 51, 753–761.

40.

Kornør

Winje

Ekeberg

Ø.

Weisæth

Kirkehei

Johansen

Steiro

(2008). Early trauma-focused cognitive-behavioural therapy to prevent chronic post-traumatic stress disorder and related symptoms: a systematic review and meta-analysis. BMC Psychiatry, 8, 81.

41.

Kovacs

(1984). The Children’s Depression, Inventory (CDI). Psychopharmacology Bulletin, 21, 995–998.

42.

Kowalik

Weller

Venter

Drachman

(2011). Cognitive behavioral therapy for the treatment of pediatric posttraumatic stress disorder: A review and meta-analysis. Journal of Behavior Therapy and Experimental Psychiatry, 42, 405–413.

43.

Macdonald

Higgins

Ramchandani

Valentine

J. C.

Bronger

L. P.

Klein

… Taylor

(2012). Cognitive behavioral interventions for children who have been sexually abused. The Cochrane Library. Retrieved from http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD001930.pub3/abstract

44.

Medical University of South Carolina. (2005). TF-CBT Web: A web based learning course for trauma focused cognitive behavioral therapy. Retrieved from http://tfcbt.musc.edu/

45.

Minami

Wampold

B. E.

Serlin

R. C.

Kircher

J. C.

Brown

G. S. J.

(2007). Benchmarks for psychotherapy efficacy in adult major depression. Journal of Consulting and Clinical Psychology, 75, 232.

46.

Newman

Weathers

F. W.

Nader

Kaloupek

D. G.

Pynoos

R. S.

Blake

D. D.

(2004). Clinician-Administered PTSD Scale for Children and Adolescents (CAPS-CA). Los Angeles, CA: Western Psychological Services.

47.

Parrish

D. E.

Rubin

(2012). Social workers’ orientations toward the evidence-based practice process: A comparison with psychologists and licensed marriage and family therapists. Social Work, 57, 201–210.

48.

Rubin

(2014). An alternative paradigm for social workers seeking to do intervention research. Social Work Research, 38, 69–71.

49.

Rubin

Bellamy

(2012). Practitioner’s guide to using research for evidence-based practice. Hoboken, NJ: John Wiley.

50.

Rubin

Parrish

D. E.

Washburn

(2014). Outcome benchmarks for adaptations of research-supported treatments for adult traumatic stress. Research on Social Work Practice. doi:10.1177/1049731514547906

51.

Rubin

(2015). Within-group effect-size benchmarks for problem-solving therapy for depression in adults. Research on Social Work Practice. doi:10.1177/1049731515592477

52.

Smith

Yule

Perrin

Tranah

Dalgleish

Clark

D. M.

(2007). Cognitive behavioral therapy for PTSD in children and adolescents: A preliminary randomized controlled trial. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 1051–1061.

53.

Scheeringa

M. S.

Weems

C. F.

Cohen

J. A.

Amaya-Jackson

Guthrie

(2011). Trauma-focused cognitive behavioral therapy for posttraumatic stress disorder in three-through six year-old children: A randomized clinical trial. Journal of Child Psychology and Psychiatry, 52, 853–860.

54.

Spielberger

C. D.

(1973). Preliminary manual for the State-Trait Anxiety Inventory for Children (“How [Feel Questionnaire”). Palo Alto, CA: Consulting Psychologists Press/

55.

Spilka

M. J.

Dobson

K. S.

(2015). Promoting the internationalization of evidence-based practice: Benchmarking as a strategy to evaluate culturally transported psychological treatments. Clinical Psychology Science and Practice, 22, 58–75.

56.

Washburn

Parrish

D. E.

(2014). Evidence based practice behaviors: Implications for third party reimbursement. Paper for the Society for Social Work Research 18th Annual Conference: Research for Social Change: Addressing Local and Global Challenges, San Antonio, Texas. January 15–19, 2014.

57.

Webb

Hayes

A. M.

Grasso

Laurenceau

J. P.

Deblinger

(2014). Trauma-focused cognitive behavioral therapy for youth: Effectiveness in a community setting. Psychological Trauma: Theory, Research, Practice, and Policy, 6, 555.

58.

Weisz

J. R.

Ugueto

A. M.

Cheron

D. M.

Herren

(2013). Evidence-based youth psychotherapy in the mental health ecosystem. Journal of Clinical Child & Adolescent Psychology, 42, 274–286.

59.

Wilson

D. B.

(2011). Calculating effect-sizes. The Campbell Collaboration. Retrieved from http://www.campbellcollaboration.org/artman2/uploads/1/2_D_Wilson__Calculating_ES.pdf