Examining Effectiveness of Group Cognitive-Behavioral Therapy for Externalizing and Internalizing Disorders in Urban Schools

Abstract

This article presents outcome data of the implementation of three group cognitive-behavioral therapy (GCBT) interventions for children with externalizing behavior problems, anxiety, and depression. School counselors and graduate students co-led the groups in two low-income urban schools. Data were analyzed to assess pre-treatment to post-treatment changes in diagnostic severity level. Results of the exploratory study indicated that all three GCBT protocols were effective at reducing diagnostic severity level for children who had a primary diagnosis of an externalizing disorder, anxiety disorder, or depressive disorder at the clinical or intermediate (at-risk) level. All three GCBT protocols were implemented with relatively high levels of fidelity. Data on the effectiveness of the interventions for reducing diagnostic severity level for externalizing and internalizing spectrum disorders and for specific disorders are presented. A discussion of implementation of mental health evidence-based interventions in urban schools is provided.

Keywords

implementation urban schools group CBT effectiveness diagnostic severity level

Schools are one of the main venues for the delivery of mental health services to children (Rones & Hoagwood, 2000) and may be the ideal context in which to implement evidence-based interventions (EBIs; Eiraldi, Benjamin Wolk, Locke, & Beidas, 2015). Services are offered in convenient locations and are provided at little or no cost to families (Taras, 2004). This is important because low-income and ethnically diverse children lag well behind their middle class, Caucasian counterparts in rates of service utilization (Cummings, Ponce, & Mays, 2010). School-based services reduce the stigma associated with seeking mental health services and also afford the opportunity to serve children who are at risk of mental disorders (Taras, 2004). In addition, benefits include the ability to implement interventions in the environment in which most symptoms are triggered (Masia-Warner et al., 2005) and to incorporate protocol specific interventions, with peer and teacher involvement as needed for generalizability (Mychailyszyn et al., 2011).

Unfortunately, research suggests that services provided in low-income urban schools result in little to no effect in child outcomes (Farahmand, Grant, Polo, Duffy, & DuBois, 2011). For example, in a review of interventions conducted at the selected or indicated prevention level, the effect size for internalizing disorders was 0.20, and the effect size for externalizing disorders and substance use was −0.14 (Farahmand et al., 2011). The disappointing results of some EBI dissemination studies, especially in published school-based treatment studies (Sanetti, Gritter, & Dobey, 2011), have been linked to inadequate training of therapists and poor treatment fidelity (Weist et al., 2014). The purpose of this exploratory study is to present findings on the effectiveness of three group cognitive-behavioral therapy (GCBT) interventions for children with or at risk of externalizing behavior problems (Coping Power Program [CPP]; Lochman, Wells, & Lenhart, 2008), anxiety (Friends for Life [FRIENDS]; Barrett, 2008), and depression (Primary and Secondary Control Enhancement Training [PASCET]; Connor-Smith, Polo, Jensen Doss, & Weisz, 2004) in two urban schools. Trained school counselors and graduate students serving as co-therapists conducted the interventions in two low-income urban schools within the context of a school-wide positive behavioral interventions and supports (SWPBIS) program development project funded by the Centers for Disease Control and Prevention (CDC).

Mental Health Problems in Schools

Aggressive, defiant, disruptive, and antisocial behavior such as the behavior seen in children with, or at risk of, externalizing behavior disorders (i.e., oppositional defiant disorder [ODD], conduct disorder [CD], attention-deficit/hyperactivity disorder [ADHD]) are highly prevalent in urban school settings (Furlong, Morrison, & Jimerson, 2007). These disorders have been found to lead to academic underachievement, grade retention, school suspension and expulsion, and later problems with the law (Frick, 1998; Ma, Phelps, Lerner, & Lerner, 2009). Early onset of aggressive and antisocial behaviors in elementary school has been found to be related to a persistent and chronic trajectory of antisocial behavior into middle childhood and adulthood (Ettekal & Ladd, 2015; Wildeboer et al., 2015).

Anxiety and depression affect 10% to 12% of children in the United States (Beesdo, Knappe, & Pine, 2009; Merikangas, Nakamura, & Kessler, 2009). These disorders are highly prevalent among inner-city school children and often go unidentified and untreated (Pina & Silverman, 2004). Children with these disorders are much more likely than their peers to have problems with peer and parent–child relations (Bergeron et al., 2007; Greco & Morris, 2005), academic achievement (Fergusson & Woodward, 2002; Van Amerigen, Manicini, & Farvolden, 2003), school refusal (Kearney & Albano, 2004), and future socioemotional adjustment (Birmaher, Ryan, Williamson, Brent, & Kaufman, 1996). School factors, such as peer problems, academic pressures, and school violence, contribute to and exacerbate symptoms (Rones & Hoagwood, 2000). Sequelae of untreated childhood anxiety and depression include chronic anxiety, depression, and substance abuse in adulthood (Birmaher, Ryan, Williamson, Brent, Kaufman, et al., 1996; Costello, Mustillo, Erkanli, Keeler, & Angold, 2003).

Effective Interventions in the School Setting

Within the school setting, children can be grouped into three relatively distinct populations: (a) typically developing students; (b) those at risk of behavioral, emotional, or academic problems; and (c) students with behavioral, emotional, or academic problems (Sprague et al., 2002). Multi-tiered approaches to prevention (e.g., universal, selective, indicated) can be effective in reducing behavioral and emotional problems in urban schools (Cook et al., 2015).

Individual and group CBT are effective treatments for externalizing behavior problems, anxiety, and depression in school settings (e.g., Benjamin, Taylor, Goodin, & Creed, 2014; Lochman & Wells, 2002b). Group CBT is feasible to implement in schools and less expensive than individual CBT and can be delivered at the selected and indicated level in underresourced urban schools (Ginsburg, Becker, Kingery, & Nichols, 2008). The three group interventions used in this study are examples of effective programs for the school setting.

CPP has been found to be effective at reducing aggressive behavior, covert delinquent behavior, and substance abuse among aggressive boys with gains maintained at 1-year follow-up (Lochman & Wells, 2004). Growth curve analyses showed that CPP had linear effects through the 3 years after intervention on reductions in aggressive behavior and academic behavior problems (Lochman, Wells, Qu, & Chen, 2013).

FRIENDS (Barrett, 2008) has been shown to be effective for the prevention and treatment of anxiety disorders (Briesch, Sanetti, & Briesch, 2010). For example, in a randomized trial with children diagnosed with an anxiety disorder, 69% of children assigned to FRIENDS were diagnosis free at the end of the 10-week trial compared with 6% of children assigned to a wait-list condition (Shortt, Barrett, & Fox, 2001). It has been shown that FRIENDS is an effective school-based intervention for at-risk children as well as for children who meet diagnostic criteria for an anxiety disorder (Briesch et al., 2010).

PASCET (Bearman, Ugueto, Alleyne, & Weisz, 2010; Weisz, Thurber, Sweeney, Proffitt, & LeGagnoux, 1997) is an effective treatment for mild to moderate forms of depression in children. In an efficacy trial, children in the PASCET condition were more likely to transition from above the normal range for depressive symptoms to within the normal range at post-treatment (50% vs. 16%) and 9-month follow-up (62% vs. 31%) than children in a control group (Weisz et al., 1997).

The treatment manuals for CPP, FRIENDS, and PASCET include components for the teaching of skills highlighted in the social emotional literature (see Weissberg, Durlak, Domitrovich, & Gullotta, 2015) as being important for behavioral and emotional adjustment. These include self-awareness (e.g., recognize how thoughts, feelings, and actions are interconnected), social awareness (e.g., perspective taking), coping skills (e.g., problem solving, relaxation), and social skills (e.g., handling peer pressure, assertiveness, seeking help when needed). This study examines whether low-income urban children with, or at risk for, the development of externalizing and internalizing psychiatric disorders improve after participating in group interventions in which these skills are taught.

Implementing EBIs in the Urban School Context

Clinicians who provide services in underresourced schools, such as counselors, social workers, and school psychologists, often lack adequate training and supervision on delivering EBIs (Olin, Saka, Crowe, Forman, & Hoagwood, 2009). When and if supervision is provided, it typically does not provide sufficient support for implementation of EBIs (Accurso, Taylor, & Garland, 2011; Power, Manz, & Leff, 2003). This study addresses this limitation by providing support to school-based clinicians in two ways: (a) providing consultation on session preparation and (b) pairing staff with psychology graduate students in co-therapy format. A different but also important challenge in implementing EBIs in underresourced urban schools is the difficulty associated with providing services to children with multiple comorbidities. Indeed, the presence of multiple comorbidities among low-income urban children in need of mental health services is the norm rather than the exception (Garland et al., 2001; Hogue & Dauber, 2013). An important task for school mental health researchers is to determine the effectiveness of EBIs for treating target disorders as well as comorbid disorders within the externalizing and internalizing spectrum.

Purpose of the Study

This article presents data gathered in the context of a SWPBIS program development project. SWPBIS is a multi-tiered systems approach based on the public health model of prevention and is designed to reduce the risks of behavior problems. Studies are needed to assess the effectiveness of EBIs for moving children down the risk continuum from having a disorder to being at risk of a disorder and to no longer meeting criteria for a disorder (Sprague & Walker, 2000; Sprague et al., 2002).

The main aim of the study was to examine the effectiveness of GCBT in the urban school setting for the most common externalizing and internalizing conditions. Given the high rate of comorbidities in the urban school population, another aim of the study was to assess the effectiveness of interventions for target disorders and for comorbid disorders. We examined the following research questions:

Research Question 1: Does participation in GCBT interventions for externalizing disorders (CPP) and internalizing disorders (FRIENDS for anxiety disorders, and PASCET for depressive disorders) reduce diagnostic severity at post-treatment for broad-spectrum categories (e.g., externalizing disorders in general) and for specific disorders within the broad spectrum?

Research Question 2: Given the high comorbidity between anxiety and depression, is FRIENDS effective for depressive disorders and PASCET effective for anxiety disorders?

Research Question 3: What percentage of students in CPP, FRIENDS, and PASCET demonstrate a reduction in diagnostic severity level and what percentage of children show an increase in diagnostic severity level?

Research Question 4: Can CPP, FRIENDS, and PASCET be implemented with fidelity?

In addition, we examined changes in diagnostic severity at a 3-month follow-up for a smaller subset of children. We hypothesized that participation in each of the programs would lead to a reduction in diagnostic severity level for broad-spectrum categories and for specific disorders targeted by each program. Given previous findings, we expected that FRIENDS would lead to reductions in diagnostic severity level for depressive disorders. Given lack of prior evidence, no hypotheses were made regarding the effectiveness of PASCET for anxiety disorders. We expected that all three programs would be implemented with relatively high levels of fidelity.

Method

Participants and Setting

The study was conducted in two K-8 inner-city public schools situated in a large city in the Northeast section of the United States. School A served 648 students (75% Latino, 18% African American, 1% Caucasian, 1% Asian, 5% Other). School B served 1,134 students (65% Latino, 16% African American, 11% Asian, 4% Caucasian, 4% Other). One hundred percent of students in both schools were eligible for free or subsidized lunch.

One hundred fourteen children (63% male) participated in one of the three group interventions over a span of 3 years. There were no differences between the groups regarding gender and ethnic composition. Children were grouped according to developmental level; children in Grades 4 to 6 and Grades 7 and 8 were assigned to a younger and an older group, respectively. Typically, three to five children were assigned to each group. Children who were absent for a session received an individual make-up session. The group sessions were conducted in the school setting at a time that did not interfere with academic instruction (e.g., free period or lunch period). All sessions were video-recorded to assess for implementation fidelity. All pertinent investigational review boards approved the study.

Measures

Changes in diagnostic status

Parents were interviewed in English (n= 74, 65%) or in Spanish (n = 40, 35%) via the NIMH Diagnostic Interview Schedule for Children, Computer Version, 4th Edition (NIMH C-DISC-IV), for 15 disorders: externalizing/disruptive behavior disorders (e.g., ADHD, conduct, and oppositional defiant), anxiety disorders (e.g., social phobia, separation anxiety, specific phobia, panic, agoraphobia, generalized anxiety, selective mutism, obsessive-compulsive disorder [OCD], and post-traumatic stress disorder [PTSD]), and mood disorders (e.g., major depressive episode, dysthymia, manic/hypomanic episode). The NIMH C-DISC-IV (Shaffer et al., 1996; Shaffer, Fisher, Lucas, Dulcan, & Schwab-Stone, 2000) is a highly structured, diagnostic interview with good psychometric properties that is commonly used in epidemiologic and clinical studies. There are no significant differences between the English and Spanish versions of the instrument with regard to content or psychometric properties (Bravo et al., 2001). The structured nature of the interview does not allow for subjective interpretation, therefore eliminating the need for diagnostic reliability checks or interrater reliability checks (Shaffer et al., 2000). Most questions are brief, containing no more than two concepts (e.g., a time period and a symptom description), and require a “yes” or “no” response. Very few questions allow a “sometimes” or open-ended response. The instrument reports three levels of diagnostic severity for each disorder: Positive, Intermediate (at-risk), or Negative. The NIMH C-DISC-IV was administered at pre-, post-, and 3-month follow-up.

In addition to the C-DISC-IV results, the study also utilized an Interference Thermometer (IT; Silverman & Albano, 1996), from the parent to indicate the degree to which each disorder, endorsed at the positive or intermediate level, interfered with the child’s functioning. The IT has a 9-point scale (0 = none; 8 = a lot) with higher scores indicating more severity. It is used to determine primary and secondary diagnoses. The IT was originally developed for children with anxiety disorders but was modified for this study to include children who present with externalizing behavior problems and symptoms of depression.

Fidelity

Group sessions were video-recorded to assess for content fidelity (i.e., the ability of therapists to deliver the content of each session as specified in the manual). To measure fidelity, two independent coders completed a Fidelity Checklist (FC) after viewing approximately 25% of the video-recording of a session chosen at random. The FC listed program components for each session. A “yes” or “no” response was used to indicate whether a content area was covered.

Group Assignment

School counselors and project staff conducted in-service training with teachers on how to recognize children who might have problems with externalizing behavior, anxiety, and depression and how to go about referring children to the groups. School staff identified children for possible participation in the study following standard procedures for children in need of services. Children who had exhibited behavioral or emotional problems in the classroom were referred following the school district’s Comprehensive Student Assistance Process (CSAP) in which the referral is discussed by the school counselors and other members of the CSAP. If members of the CSAP agreed that the children met inclusion criteria, they were referred to study staff for an eligibility evaluation. School staff contacted the parents initially to provide a brief overview of the study. They also obtained verbal consent for research staff to contact them to provide a more thorough description of the study to the parents and to obtain written consent. Parents gave informed consent and children gave assent. Children who met primary positive or intermediate diagnostic criteria for an externalizing disorder based on the NIMH C-DISC IV and IT scale were assigned to CPP. Children who met primary positive or intermediate diagnostic criteria for an anxiety disorder were assigned to FRIENDS. Those who met primary positive or intermediate diagnostic criteria for a depressive disorder were assigned to PASCET. Once the groups started, parents received periodic updates regarding the progress children were making in the groups.

Interventions

CPP (Curry, Wells, Lochman, Craighead, & Nagy, 2003; Lochman & Wells, 2002a, 2004) is a social-cognitive, multi-component group intervention for elementary and middle school students at risk of externalizing behavior disorders. In addition to anger management, CPP includes units on goal setting, emotional awareness, relaxation training, social skills training, problem solving, and handling peer pressure (Lochman, Powell, Boxmeyer, Deming, & Young, 2007). CPP offers eight sessions in the first year of intervention and 25 sessions in the second year of intervention. Most of the content is taught during the first eight sessions. Studies using an earlier, briefer (12 session) version of CPP (Anger Coping) reported significant reductions in aggressive behavior at post-intervention among targeted aggressive boys, compared with untreated aggressive boys and normal controls (Lochman, 1985; Lochman & Curry, 1986). For this study, we provided CPP in 12 weekly, 45-min sessions to make CPP more feasible for implementation in underresourced urban schools. In making program adaptations, we preserved all main components of the protocol.

FRIENDS addresses physiological, learning/behavioral, and cognitive processes that interact in the development and perpetuation of excessive anxiety (Barrett, 1998). These processes include (a) understanding the physical expression of anxiety and using relaxation skills; (b) problem-solving skills, graded exposure, and self-reward; and (c) cognitive restructuring and self-talk. The FRIENDS protocol consists of 10 weekly 70-min sessions and two booster sessions. For the present study, we included the booster sessions in the regular protocol and reduced the length of each session to fit them into the typical class period (45 min) for a total of 12 sessions.

PASCET was developed taking into account “real-world” implementation contexts (Szigethy et al., 2007; Weisz et al., 2009). The group version (Bearman et al., 2010) addresses symptoms of mild to moderate depression by teaching coping skills through the use of video-recorded vignettes and group discussion. Through the video-recordings, youngsters learn nine coping skills that can be used to gain control of their mood when stressful events make them feel sad or upset. Two primary control skills are emphasized: (a) identifying and consciously engaging in activities that the child finds mood enhancing and (b) skill building through goal setting and practice in activities that the child values. Three secondary control skills are emphasized: (a) identifying and modifying depressive thoughts, (b) cognitive techniques for mood enhancement, and (c) relaxation and positive imagery. PASCET was delivered in 12, 45-min sessions.

Training and Consultation

External consultants and project staff conducted an initial workshop with school counselors and members of the research team on intervention implementation. Trainers associated with the developers of FRIENDS and PASCET provided training on those protocols. Project staff who had previously received training by the developers of CPP conducted training for CPP. The training structure consisted of a one-and-a-half-day workshop for each program that included discussion of the theoretical background (identification of symptoms, prevalence rates, treatment efficacy), the development of each intervention (theoretical rationale, key components, efficacy/effectiveness findings), and a detailed review of the intervention sessions (content, structure, process, implementation challenges). Training included both didactics and active learning activities such as small group discussions, role-plays, behavior rehearsals, watching video-recorded sessions, and demonstration of techniques.

Following the initial training, members of the research team conducted weekly consultation with all staff participants on content delivery and implementation barriers. The principal investigator and two postdoctoral fellows in applied child psychology led the consultation sessions. The consultants, two of whom were English/Spanish bilingual, provided support to the counselors by examining intervention progress, addressing implementation barriers using problem-solving strategies, and preparing for the next sessions. Counselors were involved in decisions about every aspect of content delivery, including preparing handouts, determining of how much time to dedicate to each section of the manual, and determining how many make-up sessions to be held for children who missed some of the sessions.

Data Analysis

The data analysis strategy focused on assessing whether the three group interventions were effective at decreasing level of diagnostic severity (i.e., from clinical to intermediate, clinical to negative, and intermediate to negative) for children who had post-treatment data as measured by the NIMH C-DISC-IV. We wanted to assess whether CPP was an effective treatment for externalizing disorders in general and for ODD, CD, and ADHD, specifically. Moreover, we wanted to determine whether FRIENDS was an effective intervention for internalizing disorders in general, and separation anxiety disorder (SAD), social phobia (SOPHOB), and generalized anxiety disorder (GAD), in particular. These disorders share an underlying construct of anxiety which is somewhat distinct from other anxiety disorders, such as obsessive-compulsive disorder (OCD) and specific phobias (Beesdo et al., 2009). Finally, we wanted to assess whether PASCET was an effective intervention for internalizing disorders in general, and major depressive episode (MDE), specifically.

Prior to statistical analysis, the diagnostic status of each participant at each time point (pre-, post-, and follow-up) was determined using the following ordinal scale: 0 = Negative (no diagnosis), 1 = Intermediate (at risk of diagnosis), 2 = Positive (presence of diagnosis). In evaluating the effect of each intervention on specific disorder status, a 2 × 2 McNemar test was utilized to compare pre and post diagnostic status and pre and follow-up status. At pre-intervention, participants were grouped as either not having the diagnosis or being at risk or positive for a diagnosis. At post-intervention, participants were differentiated by whether their diagnostic status remained the same as baseline (yes) or changed (no). Crossing pre-intervention and post-intervention status created four cells: (a) no change for positive or intermediate diagnostic status, (b) no change in negative diagnostic status, (c) decline in diagnostic status (i.e., from negative to intermediate status, or from intermediate to positive status), and (d) improvement in diagnostic status (i.e., from intermediate to negative, or from positive to intermediate).

In evaluating the effect of each intervention on a broad spectrum of disorders, a total score was derived at pre-intervention, post-intervention, and follow-up by summing the scores obtained for each specific disorder within the broad spectrum (e.g., ODD, CD, and ADHD for the externalizing disorders). The sum of scores at pre-intervention was crossed with the sum scores at post-intervention to create a 2 × 2 table for McNemar test comparisons.

Given the high comorbidity level between anxiety and depressive disorders, we examined whether FRIENDS was effective for children diagnosed with MDE. We also examined whether PASCET was effective for children diagnosed with anxiety disorders. Due to the exploratory nature of this study, significance was examined at p < .05. Changes from pre-intervention to follow-up are presented using descriptive statistics due to the small sample sizes.

Content fidelity was scored as a sum of covered content (i.e., a “yes” response) divided by total content to be covered per session. Total fidelity for the group was the sum of fidelity levels for all sessions divided by the total number of sessions. Total fidelity for the program was the average fidelity for all groups. Interrater reliability between two independent coders was assessed for 25% of the sessions chosen at random. The assessment of interrater reliability was evaluated using kappa coefficients, which represents agreement between two observers taking into account the fact that observers sometimes agree or disagree simply by chance (Viera & Garrett, 2005).

Results

Attendance and Fidelity

Average number of regularly scheduled group sessions attended by participants was 6.4 (3.4 SD) for CPP, 6.8 (2.7 SD) for FRIENDS, and 5.6 (4.0 SD) for PASCET. School counselors and project staff reviewed material with absent participants individually, or in groups of two, prior to the upcoming session. Using this procedure, 83% of participants in CPP, 85% in FRIENDS, and 82% in PASCET received the entire content of the interventions.

Fidelity data revealed that 88% of the elements of CPP were implemented as intended. Interrater reliability between the two raters for CPP was κ = .43 (p < .001), 95% confidence intervals [CI] = [0.353, 0.497]. Fidelity for FRIENDS was 87% and interrater reliability was κ = .63 (p < .001), 95% CI = [0.58, 0.68]. Fidelity for PASCET was 94% and agreement between the two raters was κ = .63 (p < .001), 95% CI = [0.55, 0.72]. The kappa statistics across the three groups indicated a moderate degree of agreement between the raters (Viera & Garrett, 2005).

Diagnostic Severity Outcomes

CPP

Fifty-seven children were enrolled in CPP. There were 37 children (65%) with post-treatment data. A McNemar’s test was conducted to evaluate whether there was a change in diagnostic status for one or more of the three externalizing disorders at post-intervention. The results revealed a significant decrease in diagnostic risk status from pre- to post-treatment, exact McNemar test = 9.14 (1), p = .004. Table 1 shows that six children (16%) increased their level of diagnostic severity at post-treatment, nine children (24%) maintained the same level of diagnostic severity, and 22 children (60%) improved (i.e., changed from a positive to intermediate or no diagnosis, or changed from an intermediate to no diagnosis).

Table 1.

Pre- to Post-Treatment Changes in Diagnostic Severity Level for Children Participating in CPP.

Pre-treatment			Post-treatment			Pre- to post- change
Externalizing code	Frequency	%	Externalizing code	Frequency	%	Externalizing change	Frequency	%
0	3	5.26	0	6	16.22	−2	1	2.70
1	10	17.54	1	5	13.51	−1	5	13.51
2	9	15.79	2	5	13.51	0	9	24.32
3	11	19.30	3	9	24.32	1	15	40.54
4	15	26.32	4	7	18.92	2	4	10.81
5	5	8.77	5	2	5.41	3	1	2.70
6	4	7.02	6	3	8.11	4	1	2.70
						5	1	2.70
ODD code	Frequency	%	ODD code	Frequency	%	ODD change	Frequency	%
Negative	10	17.54	Negative	9	24.32	−1	4	10.81
Intermediate	17	29.82	Intermediate	8	21.62	0	21	56.76
Positive	30	52.63	Positive	20	54.05	1	10	27.03
						2	2	5.41
CD code	Frequency	%	CD code	Frequency	%	CD change	Frequency	%
Negative	41	71.93	Negative	32	86.49	−1	2	5.41
Intermediate	8	14.04	Intermediate			0	27	72.97
Positive	8	14.04	Positive	5	13.51	1	4	10.81
						2	4	10.81
ADHD code	Frequency	%	ADHD code	Frequency	%	ADHD change	Frequency	%
Negative	14	24.56	Negative	11	29.73	−2	1	2.70
Intermediate	17	29.82	Intermediate	12	32.43	−1	1	2.70
Positive	26	45.61	Positive	14	37.84	0	24	64.86
						1	11	29.73

Note. The table shows the frequency and percentage of participants who received a range of scores (codes) for the broad spectrum of externalizing disorders as well as the specific disorders within the spectrum (ODD, CD, ADHD), with pre-intervention and post-intervention statistics reported in separate columns. The “Pre- to post- change” column shows the net change of scores (codes) from pre-intervention to post-intervention, and the corresponding frequency and percent of change scores for the externalizing spectrum and each specific disorder. CPP = Coping Power Program; ODD = oppositional defiant disorder; CD = conduct disorder; ADHD = attention-deficit/hyperactivity disorder.

Further analysis was conducted to evaluate whether participation in CPP reduced the severity of ODD diagnostic severity level. The results revealed a statistical trend, McNemar = 4.00 (1), p = .077. After participation in CPP, three out of 37 children (8%) did not meet criteria for ODD at pre- and post-intervention, and 18 children (49%) continued to have the same level of severity. Four children (11%) demonstrated an increase in severity of ODD (i.e., change from intermediate to positive diagnosis), and 12 children (32%) had a decrease in diagnostic severity. An analysis was also conducted to evaluate whether participation in CPP reduced diagnostic severity level for children diagnosed with CD. The results revealed no statistical difference, exact McNemar test = 3.60 (1), p = .109. Finally, an analysis was conducted to evaluate whether participation in CPP reduced diagnostic severity level for children diagnosed with any subtype of ADHD. The results revealed a significant decrease, exact McNemar test = 6.23 (1), p = .023. After participation in CPP, six children (16%) did not meet diagnostic criteria for ADHD at pre- and post-intervention, and 18 children (49%) maintained the same level of diagnostic severity. Eleven children (30%) demonstrated improvement at post-intervention and no longer met criteria for ADHD, and two children (5%) demonstrated a decline and met criteria for ADHD at post-intervention. With regard to follow-up, two out of 12 children (17%) had an increase in diagnostic severity level for the broad spectrum of externalizing disorders compared with pre-treatment, three children (25%) did not change, and seven children (58%) had a decrease in diagnostic severity level.

FRIENDS

Thirty-two children participated in FRIENDS groups, and 23 (72%) had post-treatment data. All of the children in the sample met criteria having an intermediate or positive diagnosis for at least one anxiety disorder at pre-intervention. The results revealed a significant reduction in diagnostic severity level at post-intervention, exact McNemar test = 13.24 (1), p < .001. Specifically, six children (26%) demonstrated no change in diagnostic severity level, 16 children (70%) demonstrated a decrease, and one child (4%) showed an increase (see Table 2). Given that the analyses for FRIENDS included children who met criteria for mood disorders, we repeated the analyses after excluding the sum of scores for mood disorders. The results did not change, exact McNemar test = 13.24 (1), p < .001.

Table 2.

Pre- to Post-Treatment Changes in Diagnostic Severity Level for Children Participating in FRIENDS.

Pre-treatment			Post-treatment			Pre- to post- change
Internalizing code	Frequency	%	Internalizing code	Frequency	%	Internalizing change	Frequency	%
1	4	12.50	0	6	26.09	−2	1	4.35
2	5	15.63	1	3	13.04	0	6	26.09
3	3	9.38	2	6	26.09	1	4	17.39
4	7	21.88	3	1	4.35	2	4	17.39
5	4	12.50	4	4	17.39	3	2	8.70
6	1	3.13	5	1	4.35	4	2	8.70
7	3	9.38	6	1	4.35	5	1	4.35
9	2	6.25	11	1	4.35	6	3	13.04
11	2	6.25
12	1	3.13
SAD code	Frequency	%	SAD code	Frequency	%	SAD change	Frequency	%
Negative	12	37.50	Negative	12	52.17	−2	1	4.35
Intermediate	7	21.88	Intermediate	4	17.39	−1	1	4.35
Positive	13	40.63	Positive	7	30.43	0	15	65.22
						1	4	17.39
						2	2	8.70
SOPHOB code	Frequency	%	SOPHOB code	Frequency	%	SOPHOB change	Frequency	%
Negative	15	46.88	Negative	17	73.91	−1	1	4.35
Intermediate	9	28.13	Intermediate	3	13.04	0	14	60.87
Positive	8	25.00	Positive	3	13.04	1	6	26.09
						2	2	8.70
GAD code	Frequency	%	GAD code	Frequency	%	GAD change	Frequency	%
Negative	23	71.88	Negative	22	95.65	0	18	78.26
Intermediate	3	9.38	Intermediate			1	2	8.70
Positive	6	18.75	Positive	1	4.35	2	3	13.04
MDE	Frequency	%	MDE code	Frequency	%	MDE change	Frequency	%
Negative	20	62.50	Negative	17	73.91	0	17	73.91
Intermediate	8	25.00	Intermediate	5	21.74	1	6	26.09
Positive	4	12.50	Positive	1	4.35

Note. The table shows the frequency and percentage of participants who received a range of scores (codes) for the broad spectrum of internalizing disorders as well as the specific disorders within the spectrum (SAD, SOPHOB, GAD, MDE), with pre-intervention and post-intervention statistics reported in separate columns. The “Pre- to post- change” column shows the net change of scores (codes) from pre-intervention to post-intervention, and the corresponding frequency and percent of change scores for the internalizing spectrum and each specific disorder.

SAD = separation anxiety disorder; SOPHOB = social phobia; GAD = generalized anxiety disorder.

We conducted further analyses with children diagnosed with SAD, SOPHOB, and GAD. Participation in FRIENDS did not reduce the severity of SAD diagnostic level, exact McNemar test = 2.0 (1), p = .289. Changes in diagnostic severity level from pre- to post-treatment were significant for SOPHOB, exact McNemar test = 5.4 (1), p = .039. Eleven children (48%) did not meet diagnostic criteria for SOPHOB at pre- or post-intervention, and three children (13%) diagnosed with SOPHOB continued to meet criteria at post-treatment. One child (4%) got worse and eight children (35%) improved. Finally, pre- to post-treatment comparison revealed a statistical trend for GAD, exact McNemar test = 5.0 (1), p = .063. Seventeen children (74%) did not meet criteria for GAD at pre- or post-intervention, and one child (4%) continued to meet criteria at post-intervention. Five children (22%) improved and no children got worse.

Analyses were also conducted to assess whether FRIENDS was effective for children diagnosed with MDE. The results revealed a significant decrease in diagnostic severity, exact McNemar test = 6.0 (1), p = .031. Seventeen out of 23 children (74%) demonstrated no change in diagnostic severity level, six children (26%) showed improvement and no longer met diagnostic criteria at post-intervention, and no children became worse. With regard to follow-up, three of 12 children (25%) did not change over the course of intervention, six (50%) demonstrated improvement, and three (25%) demonstrated a decline in functioning.

PASCET

Twenty-five children were enrolled in PASCET. There were 22 children (88%) with post-treatment data. A McNemar test was conducted to evaluate whether there was a change in diagnostic status for one or more of the 12 internalizing disorders at post-evaluation. The results revealed a significant difference in diagnostic risk status from pre- to post-treatment, exact McNemar test = 7.14 (1), p = .013. Table 3 shows that two children (9%) increased their level of diagnostic severity at post-treatment, 8 children (36%) maintained the same level of diagnostic severity, and 12 children (55%) improved (i.e., changed from a positive to intermediate or no diagnosis, or changed from an intermediate to no diagnosis).

Table 3.

Pre- to Post-Treatment Changes in Diagnostic Severity Level for Children Participating in PASCET.

Pre-treatment			Post-treatment			Pre- to post- change
Internalizing code	Frequency	%	Internalizing code	Frequency	%	Internalizing change	Frequency	%
0	8	32.00	0	14	63.64	−1	2	9.09
1	4	16.00	1	2	9.09	0	8	36.36
2	6	24.00	2	4	18.18	1	5	22.73
3	3	12.00	3	1	4.55	2	2	9.09
4	2	8.00	4	1	4.55	3	2	9.09
6	1	4.00				4	2	9.09
7	1	4.00				6	1	4.55
MDE code	Frequency	%	MDE code	Frequency	%	MDE change	Frequency	%
Negative	12	48.00	Negative	17	77.27	−1	3	13.64
Intermediate	9	36.00	Intermediate	2	9.09	0	11	50.00
Positive	4	16.00	Positive	3	13.64	1	6	27.27
						2	2	9.09
SAD code	Frequency	%	SAD Code	Frequency	%	SAD change	Frequency	%
Negative	19	76.00	Negative	19	86.36	−1	2	9.09
Intermediate	3	12.00	Intermediate			0	17	77.27
Positive	3	12.00	Positive	3	13.64	1	1	4.55
						2	2	9.09

Note. The table shows the frequency and percentage of participants who received a range of scores (codes) for the broad spectrum of internalizing disorders as well as specific disorders within the spectrum (MDE, SAD), with pre-intervention and post-intervention statistics reported in separate columns. The “Pre- to post- change” column shows the net change of scores (codes) from pre-intervention to post-intervention, and the corresponding frequency and percent of change scores for the internalizing spectrum and each specific disorder. PASCET = Primary and Secondary Control Enhancement Training; SAD = separation anxiety disorder.

Further analyses were conducted after excluding the sum of scores for anxiety disorders. Results revealed a decrease in diagnostic severity from pre- to post-treatment, exact McNemar test = 5.33 (1), p = .039. After participation in PASCET, eight out of 22 children (36%) did not meet criteria for mood disorder at pre- and post-intervention, and two children (9%) continued to have the same level of severity. Ten children (46%) no longer met criteria for mood disorder at post-treatment, and two children (9%) demonstrated an increase in severity of mood disorder. An analysis was also conducted to evaluate whether participation in PASCET reduced severity of MDE. The results revealed no statistical difference, exact McNemar test = 2.27 (1), p = .227. Additional analyses were conducted to evaluate whether participation in PASCET reduced the severity of SAD. Data for other anxiety disorders was incomplete and therefore unavailable to interpret for analyses. After participation in PASCET, results revealed no statistical difference in diagnostic changed for children with SAD, exact McNemar test = .20 (1), p = 1.00. With regards to follow-up, eight out of 13 children (62%) had a decrease in diagnostic severity level for the broad spectrum of internalizing disorders compared with pre-treatment, five children (38%) did not change, and no children had an increase in diagnostic severity level.

Discussion

This exploratory study is unique in that it examines whether GCBT implemented in urban schools can reverse the trajectory toward increased mental health risk status among students. We tested three different GCBT packages to determine whether the proportion of children with mental health disorders decreases after participation in group treatment, and whether children had a decrease in diagnostic severity for specific disorders, as assessed by a “gold standard” diagnostic instrument. Another contribution is that the study provides estimates of percent improvement as well as decline in symptoms among study participants.

In general, the results of the study showed that GCBT can successfully be used in urban schools to reverse the trajectory toward increased risk of externalizing and internalizing disorders. Most of the participants received the entire content of the interventions by attending scheduled sessions and participating in brief make-up sessions when absent. Each of the group interventions were able to be implemented with high levels of fidelity, providing additional evidence of the feasibility of implementing these programs in urban schools. Furthermore, the results pertaining to the effectiveness of GCBT for broad-spectrum categories (i.e., externalizing and internalizing) and individual disorders within each broad category showed that the interventions were, for the most part, effective.

Analyses were conducted to examine whether FRIENDS was effective for mood disorders and whether PASCET was effective for anxiety disorders. Results showed that FRIENDS led to reductions in diagnostic severity level for MDE and PASCET did not lead to reductions in severity level for SAD.

The findings demonstrated considerable clinical significance as a large percentage of students did indeed improve over the course of intervention. For example, 59% of children in CPP demonstrated a reduction in diagnostic risk level in the broad-spectrum externalizing domain, as did 32% for children diagnosed with ODD, and 30% of children diagnosed with ADHD. In contrast, only 11% of children with ODD and 5% of those with ADHD participating in CPP had an increase in diagnostic severity level. The results for CPP are especially noteworthy, considering that the intervention used was briefer that the version validated by numerous research investigations.

Seventy percent of children in FRIENDS had a reduction in symptom severity level for the internalizing spectrum category, as did 35% of children diagnosed with SOPHOB. Similarly, 55% of children participating in PASCET had a reduction in diagnostic severity level for the internalizing spectrum category. In contrast, for FRIENDS, only 9% of children diagnosed with SAD, and 4% of children diagnosed with SOPHOB had an increase in diagnostic severity level at post-treatment, with no children diagnosed with GAD showing increased symptomatology. Finally, no children assigned to PASCET who had a diagnosis of MDE displayed an increase in diagnostic severity. These results are particularly promising given the fact that the sample included children with multiple comorbidities.

Implementation Strategy

The encouraging results with regard to implementation fidelity and child outcomes need to be considered in relation to the implementation strategy used in the study. This investigation was designed as an effectiveness and not an efficacy study, meaning that all three interventions were implemented in the “real world,” specifically, in underresourced urban schools and with children presenting multiple comorbidities. It should be noted that the implementers received high levels of implementation support, including participation in an initial training workshop, ongoing consultation on session preparation, and problem-solving barriers to implementation. In addition, graduate students, who had varying levels of exposure to EBIs, supported school counselors through co-therapy. This implementation strategy may not be feasible for most schools. However, the child outcomes represent a valid comparison for future studies that use lower levels of support for school implementers. Indeed, studies are needed to identify feasible and cost-effective implementation strategies that can achieve reasonably successful child outcomes.

Implementation Barriers

Past research has shown that there are many barriers that make it difficult for low-income minority parents to participate in research studies (Gross, Julion, & Fogg, 2001). Low parent engagement affected both recruitment (e.g., difficulty with consent and eligibility evaluations) and subsequent data collection (e.g., post-treatment and follow-up interviews). This reduced the number of evaluable participants for analyses, especially for children participating in CPP. As some studies have demonstrated, lack of parent participation in research studies conducted in underresourced schools does not have to be endemic (Gross, Breitenstein, Eisbach, Hoppe, & Harrison, 2014). Studies have shown that effective engagement strategies with low-income families include (a) acknowledging parents’ values and their expertise about their children, (b) acknowledging that they want to be good parents, (c) reinforcing parents for their efforts to change, and (d) giving them options for achieving intervention goals (Eiraldi et al., 2015; Gross et al., 2014). Motivation strategies include eliciting self-motivating change statements and identifying, developing, and executing plans for dealing with barriers to treatment adherence and continued participation in treatment sessions (Nock & Kazdin, 2005).

Implications for Clinical Practice

The study has a number of implications for clinical practice. First, GCBT can be effectively delivered in urban schools serving children from low-income families presenting externalizing and internalizing disorders. Children can be included in GCBT if they have a combination of externalizing and internalizing disorders. However, school-based clinicians should determine which disorder causes the most impairment and choose a protocol for that disorder. As demonstrated in the study, choosing a protocol for a specific disorder within a broad spectrum (i.e., externalizing or internalizing) can also lead to reductions in other comorbid disorders within the spectrum.

Most participants in the study received all 12 sessions. Children who were not present for a given group session received the content material in brief individual sessions. Clinicians planning on using one of these protocols in their schools should ensure that children who miss a session are given the opportunity to cover the material for the missed session. Finally, clinicians who use one of these treatments must ensure that the manuals are implemented with relatively high levels of fidelity.

Limitations of the Study

There are several limitations to the study. First, the clinical effectiveness of the interventions was examined without using control groups. Even though no control groups were used in the study, we showed similar effects using three different EBIs for treating three different types of psychiatric problems. Prior studies, including studies conducted in school settings, have demonstrated the efficacy and effectiveness of these treatments. Furthermore, student behavioral and emotional functioning typically worsens over the course of a school year (Kellam, Ling, Merisca, Brown, & Ialongo, 1998). The findings of this study demonstrate that group interventions generally appear to be successful in altering this expected decline. Second, the sample size for all three groups was small, raising questions about the adequacy of the sample to capture the variability among students attending similar types of schools. Nonetheless, the impact of the relatively small sample size is that it reduced the power of the analyses to detect changes in diagnostic severity level in response to intervention. It is quite possible that a larger sample size may have resulted in sufficient power to detect statistically significant change for comparisons that had alpha levels that were marginal. Third, post-treatment data were not collected for some students in the group interventions (ranging from 12% for PASCET to 35% for CPP). It is possible that attrition had an effect on the findings although the pattern of results across intervention approaches was remarkably similar. Third, only one instrument was used to measure change. Although inclusion of other methods of assessment is highly desirable, the parent-structured interview is considered the “gold standard” in clinical trials, and as such, lends credibility to study findings (Hughes et al., 2000). Fourth, counselors and graduate students conducted most of the group sessions together. We were not able to determine whether outcomes differed depending on whether counselors or graduate students conducted the groups. In addition, analyses of program effectiveness by gender and ethnic group were not conducted due to the relatively small sample size.

Conclusion

The literature on the effectiveness of mental health interventions in low-income urban schools has reported mostly disappointing outcomes (e.g., Farahmand et al., 2011). The low or negative effect sizes of some of these studies might be explained by the quality of the interventions being used and the way in which they were implemented. This study used EBIs provided in a group format. Implementers were provided relatively high levels of support, and the intervention was implemented with relatively high levels of fidelity. Student engagement in treatment was relatively high due in part to incorporation of a procedure to provide make-up sessions for missed meetings. Results suggest that EBIs for externalizing and internalizing disorders can be effectively deployed in underresourced urban schools, provided that implementation factors such as training and supervision of therapists are addressed.

Recommendations for Future Research

Research including suitable control groups and random assignment to condition are needed to verify the effectiveness of these interventions. Studies are also needed for testing various implementation strategies to identify feasible and cost-effective approaches for implementing EBIs in underresourced urban schools. For example, research is needed to find cost-effective strategies for creating sustainable internal capacity in schools and in mental health agencies that provide services in schools. One such strategy is internal clinical supervision. As the majority of clinicians and supervisors who provide services in schools have not received adequate training on EBIs and paying external consultants is not financially feasible for many school districts and agencies, efforts must be focused on finding effective ways to train clinical supervisors to become effective supervisors of EBIs. Although numerous challenges exist in transporting EBIs to urban schools, the school setting offers a tremendous opportunity to increase access to quality mental health care for underserved children.

Footnotes

Acknowledgements

We thank the counselors from the two participating schools and the graduate students who served as co-therapists.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

CDC provided funding for this study; grant number R18 MN000003 (Eiraldi, McCurdy, and Power).

Author Biographies

Ricardo Eiraldi is Associate Professor of Clinical Psychology in Pediatrics and Psychiatry at Perelman School of Medicine of the University of Pennsylvania, and Program Director, Behavioral Health in Urban Schools, The Children’s Hospital of Philadelphia. His research focuses on dissemination and implementation of evidence-based practices in nontraditional settings.

Thomas Power is Professor of School Psychology in Pediatrics and Psychiatry at Perelman School of Medicine of the University of Pennsylvania. His research focuses on behavioral health in primary care, school mental health, and the assessment and treatment of attention-deficit/hyperactivity disorder.

Billie Schwartz is a Postdoctoral Fellow in Clinical Psychology, Behavioral Health in Urban Schools, at The Children’s Hospital of Philadelphia. She received her Ph.D. in counseling psychology from the University of Miami. Research interests include integrated behavioral health, including implementation of evidence-based practices in community settings and pediatric primary care.

Jackie Keiffer is a research coordinator, Behavioral Health in Urban Schools Program at The Children’s Hospital of Philadelphia. She received a BA in psychology from Temple University and a Master’s degree in Elementary Education from Wilmington University. Research interests include increasing access to evidence-based mental health services to underserved populations.

Barry McCurdy is the Director of the Devereux Center for Effective Schools. He received his Ph.D. in school psychology from Lehigh University. Research interests include school-based prevention and intervention strategies for children with and at risk for disruptive behavior disorders.

Manju Mathen is currently a Resource Coordinator II at The Children’s Hospital of Philadelphia. She received a BS in Biology from Arcadia University and a MS in Human Resource Development from Drexel University. Her interest includes research coordination in various settings including, hospitals, primary care offices and schools.

Abbas Jawad is Associate Professor of Biostatistics in pediatrics, University of Pennsylvania Perelman School of Medicine, and Department of Pediatrics, The Children’s Hospital of Philadelphia. His research focuses on longitudinal studies and clinical trials study designs.

References

Accurso

E. C.

Taylor

R. M.

Garland

A. F.

(2011). Evidence-based practices addressed in community-based children’s mental health clinical supervision. Training and Education in Professional Psychology, 5, 88-96. doi:10.1037/A0023537

Barrett

(1998). Evaluation of cognitive-behavioral group treatments for childhood anxiety disorders. Journal of Clinical Child Psychology, 27, 459-468. doi:10.1207/s15374424jccp2704_10

Barrett

(2008). Friends for Life: Group leaders’ manual for children. Sydney, Australia: Pathways Health and Research Centre.

Bearman

S. K.

Ugueto

Alleyne

Weisz

J. R.

(2010). Adapting CBT for depression to fit diverse youths and contexts: Applying the deployment-focused model of treatment development and testing. In Weisz

J. R.

Kazdin

A. E.

(Eds.), Evidence-based psychotherapies for children and adolescents (2nd ed., pp. 466-481). New York, NY: Guilford Press.

Beesdo

Knappe

Pine

D. S.

(2009). Anxiety and anxiety disorders in children and adolescents: Developmental issues and implications for DSM-V. Psychiatric Clinics of North America, 32, 483-524. doi:10.1016/j.psc.2009.06.002

Benjamin

C. L.

Taylor

K. P.

Goodin

Creed

T. A.

(2014). Dissemination and implementation of cognitive therapy for depression in schools. In Beidas

R. S.

Kendall

P. C.

(Eds.), Dissemination and implementation of evidence-based practices in child and adolescent health care (pp. 277-293). New York, NY: Oxford University Press.

Bergeron

Valla

J. P.

Smolla

Piche

Berthiaume

St-Georges

(2007). Correlates of depressive disorders in the Quebec general population 6 to 14 years of age. Journal of Abnormal Child Psychology, 35, 459-474. doi:10.1007/s10802-007-9103-x

Birmaher

Ryan

N. D.

Williamson

D. E.

Brent

D. A.

Kaufman

(1996). Childhood and adolescent depression: A review of the past 10 years. Part II. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 1575-1583. doi:10.1097/00004583-199612000-00008

Birmaher

Ryan

N. D.

Williamson

D. E.

Brent

D. A.

Kaufman

Dahl

R. E.

. . . Nelson

(1996). Childhood and adolescent depression: A review of the past 10 years. Part I. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 1427-1439. doi:10.1097/00004583-199611000-00011

10.

Bravo

Ribera

Rubio-Stipec

Canino

Shrout

Ramirez

. . . Martinez Taboas

(2001). Test-retest reliability of the Spanish version of the Diagnostic Interview Schedule for Children (DISC-IV). Journal of Abnormal Child Psychology, 29, 433-444. doi:10.1023/A:1010499520090

11.

Briesch

A. M.

Sanetti

L. M. H.

Briesch

J. M.

(2010). Reducing the prevalence of anxiety in children and adolescents: An evaluation of the evidence base for the FRIENDS for life program. School Mental Health, 2, 155-165. doi:10.1007/s12310-010-9042-5

12.

Connor-Smith

Polo

Jensen Doss

Weisz

J. R.

(2004). Leader’s manual: Act and Adapt video-guided PASCET program for prevention and treatment of youth depression. Los Angeles: University of California.

13.

Cook

C. R.

Frye

Slemrod

Lyon

A. R.

Renshaw

T. L.

Zhang

(2015). An integrated approach to universal prevention: Independent and combined effects of PBIS and SEL on youths’ mental health. School Psychology Quarterly, 30, 166-183. doi:10.1037/spq0000102

14.

Costello

E. J.

Mustillo

Erkanli

Keeler

Angold

(2003). Prevalence and development of psychiatric disorders in childhood and adolescence. Archives of General Psychiatry, 60, 837-844. doi:10.1001/archpsyc.60.8.83760/8/837

15.

Cummings

J. R.

Ponce

N. A.

Mays

V. M.

(2010). Comparing racial/ethnic differences in mental health service use among high-need subpopulations across clinical and school-based settings. Journal of Adolescent Health, 46, 603-606. doi:10.1016/j.jadohealth.2009.11.221

16.

Curry

J. F.

Wells

K. C.

Lochman

J. E.

Craighead

W. E.

Nagy

P. D.

(2003). Cognitive-behavioral intervention for depressed, substance-abusing adolescents: Development and pilot testing. Journal of the American Academy of Child & Adolescent Psychiatry, 42, 656-665. doi:10.1097/01.CHI.0000046861.56865.6C

17.

Eiraldi

Benjamin Wolk

Locke

Beidas

(2015). Clearing hurdles: The challenges of implementation of mental health evidence-based practices in under-resourced schools. Advances in School Mental Health Promotion, 8, 124-145. doi:10.1080/1754730X.2015.1037848

18.

Ettekal

Ladd

G. W.

(2015). Developmental pathways from childhood aggression-disruptiveness, chronic peer rejection, and deviant friendships to early-adolescent rule breaking. Child Development, 86, 614-631. doi:10.1111/cdev.12321

19.

Farahmand

F. K.

Grant

K. E.

Polo

A. J.

Duffy

S. N.

DuBois

D. L.

(2011). School-based mental health and behavioral programs for low-income, urban youth: A systematic and meta-analytic review. Clinical Psychology Science and Practice, 18, 372-390. doi:10.1111/j.1468-2850.2011.01265.x

20.

Fergusson

D. M.

Woodward

L. J.

(2002). Mental health, educational, and social role outcomes of adolescents with depression. Archives of General Psychiatry, 59, 225-231. doi:10.1001/archpsyc.59.3.225

21.

Frick

P. J.

(1998). Conduct disorders and severe antisocial behavior. New York, NY: Plenum.

22.

Furlong

M. J.

Morrison

G. M.

Jimerson

S. R.

(2007). Externalizing behaviors of aggression and violence and the school context. In Rutherford

R. B.

Jr. Quinn

M. M.

Mathur

S. R.

(Eds.), Handbook of research in emotional and behavioral disorders (pp. 243-261). New York, NY: Guilford Press.

23.

Garland

A. F.

Hough

R. L.

McCabe

K. M.

Yeh

Wood

P. A.

Aarons

G. A.

(2001). Prevalence of psychiatric disorders in youths across five sectors of care. Journal of the American Academy of Child & Adolescent Psychiatry, 40, 409-418. doi:10.1097/00004583-200104000-00009

24.

Ginsburg

G. S.

Becker

K. D.

Kingery

J. N.

Nichols

(2008). Transporting CBT for Childhood Anxiety Disorders into inner-city schools-based mental health clinics. Cognitive and Behavioral Practice, 15, 148-158. doi:10.1016/j.cbpra.2007.07.001

25.

Greco

Morris

(2005). Factors influencing the link between social anxiety and peer acceptance: Contributions of social skills and close friendships during middle childhood. Behavior Therapy, 36, 197-205. doi:10.1016/S0005-7894(05)80068-1

26.

Gross

Breitenstein

Eisbach

Hoppe

Harrison

(2014). Promoting mental health in early childhood programs: Serving low-income ethnic minority families. In Weist

M. D.

Lever

N. A.

Bradshaw

C. P.

Owens

J. S.

(Eds.), Handbook of school mental health: Research, training, practice, and policy (2nd ed., pp. 119-130). New York, NY: Springer.

27.

Gross

Julion

Fogg

(2001). What motivates participation and drop out among low-income urban families of color in a prevention intervention? Family Relations, 50, 246-254. doi:10.1111/j.1741-3729.2001.00246

28.

Hogue

Dauber

(2013). Diagnostic profiles among urban adolescents with unmet treatment needs: Comorbidity and perceived need for treatment. Journal of Emotional and Behavioral Disorders, 21, 18-32. doi:10.1177/1063426611407500

29.

Hughes

C. W.

Rintelmann

Mayes

Emslie

G. J.

Pearson

Rush

A. J.

(2000). Structured interview and uniform assessment improves diagnostic reliability. Journal of Child and Adolescent Psychopharmacology, 10, 119-131. doi:10.1089/cap.2000.10.119

30.

Kearney

C. A.

Albano

A. M.

(2004). The functional profiles of school refusal behavior. Diagnostic aspects. Behavior Modification, 28, 147-161. doi:10.1177/0145445503259263

31.

Kellam

S. G.

Ling

Merisca

Brown

C. H.

Ialongo

(1998). The effect of the level of aggression in the first grade classroom on the course and malleability of aggressive behavior into middle school. Development and Psychopathology, 10, 165-185.

32.

Lochman

J. E.

(1985). Effects of different treatment lengths in cognitive behavioral interventions with aggressive boys. Child Psychiatry & Human Development, 16, 45-56. doi:10.1007/BF00707769

33.

Lochman

J. E.

Curry

(1986). Effects of social problem-solving training and self-instruction training with aggressive boys. Journal of Clinical Child Psychology, 15, 159-164. doi:10.1207/s15374424jccp1502_8

34.

Lochman

J. E.

Powell

Boxmeyer

Deming

A. M.

Young

(2007). Cognitive-behavior group therapy for angry and aggressive youth. In Chrisner

Stewart

Freeman

(Eds.), Handbook of cognitive-behavior group therapy with children and adolescents: Specific settings and presenting problems (pp. 333-348). New York, NY: Routledge.

35.

Lochman

J. E.

Wells

K. C.

(2002a). Contextual social-cognitive mediators and child outcome: A test of the theoretical model in the Coping Power program. Development and Psychopathology, 14, 945-967. doi:10.1017/S0954579402004157

36.

Lochman

J. E.

Wells

K. C.

(2002b). The Coping Power program at the middle-school transition: Universal and indicated prevention effects. Psychology of Addictive Behaviors, 16(4 Suppl.), S40-S54. doi:10.1037/0893-164X.16.4S.S40

37.

Lochman

J. E.

Wells

K. C.

(2004). The coping power program for preadolescent aggressive boys and their parents: Outcome effects at the 1-year follow-up. Journal of Consulting and Clinical Psychology, 72, 571-578. doi:10.1037/0022-006X.72.4.571

38.

Lochman

J. E.

Wells

K. C.

Lenhart

(2008). Coping power child group program: Facilitator guide. New York, NY: Oxford University Press.

39.

Lochman

J. E.

Wells

K. C.

Chen

(2013). Three year follow-up of coping power intervention effects: evidence of neighborhood moderation? Prev Sci, 14(4), 364-376. doi: 10.1007/s11121-012-0295-0

40.

Phelps

Lerner

J. V.

Lerner

R. M.

(2009). Academic competence for adolescents who bully and who are bullied: Findings from the 4-H study of positive youth development. Journal of Early Adolescence, 29, 862-897. doi:10.1177/0272431609332667

41.

Masia-Warner

Klein

R. G.

Dent

H. C.

Fisher

P. H.

Alvir

Albano

A. M.

Guardino

(2005). School-based intervention for adolescents with social anxiety disorder: Results of a controlled study. Journal of Abnormal Child Psychology, 33, 707-722. doi:10.1007/s10802-005-7649-z

42.

Merikangas

K. R.

Nakamura

E. F.

Kessler

R. C.

(2009). Epidemiology of mental disorders in children and adolescents. Dialogues in Clinical Neuroscience, 11, 7-20.

43.

Mychailyszyn

M. P.

Beidas

R. S.

Benjamin

C. L.

Edmunds

J. M.

Podell

J. L.

Cohen

J. S.

Kendall

P. C.

(2011). Assessing and treating child anxiety in schools. Psychology in the Schools, 48, 223-232.

44.

Nock

M. K.

Kazdin

A. E.

(2005). Randomized controlled trial of a brief intervention for increasing participation in parent management training. Journal of Consulting and Clinical Psychology, 73, 872-879. doi:10.1037/0022-006X.73.5.872

45.

Olin

S. S.

Saka

Crowe

Forman

S. G.

Hoagwood

K. E.

(2009). Implementation of evidence-based interventions in schools: Issues and challenges in social-emotional learning and mental health programs. In Rosenfield

Berninger

V. W.

(Ed.), Implementing evidence-based academic interventions in school settings (pp. 253-283). New York, NY: Oxford University Press.

46.

Pina

A. A.

Silverman

W. K.

(2004). Clinical phenomenology, somatic symptoms, and distress in Hispanic/Latino and European American youths with anxiety disorders. Journal of Clinical Child and Adolescent Psychology, 33, 227-236. doi:10.1207/s15374424jccp3302_3

47.

Power

T. J.

Manz

P. H.

Leff

S. S.

(2003). Training for effective practice in the schools. In Weist

M. D.

Evans

S. W.

Lever

N. A.

(Eds.), Handbook of school mental health: Advancing practice and research (pp. 257-273). New York, NY: Kluwer Academic/Plenum.

48.

Rones

Hoagwood

(2000). School-based mental health services: A research review. Clinical Child and Family Psychology Review, 3, 223-241. doi:10.1023/A:1026425104386

49.

Sanetti

L. M. H.

Gritter

K. L.

Dobey

L. M.

(2011). Treatment integrity of interventions with children in the school psychology literature from 1995 to 2008. School Psychology Review, 40, 72-84. doi:10.1177/1098300711405853

50.

Shaffer

Fisher

Dulcan

M. K.

Davies

Piacentini

Schwab-Stone

M. E.

. . . Regier

D. A.

(1996). The NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC-2.3): Description, acceptability, prevalence rates, and performance in the MECA Study. Methods for the Epidemiology of Child and Adolescent Mental Disorders Study. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 865-877. doi:10.1097/00004583-199607000-00012

51.

Shaffer

Fisher

Lucas

C. P.

Dulcan

M. K.

Schwab-Stone

M. E.

(2000). NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): Description, differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy of Child & Adolescent Psychiatry, 39, 28-38. doi:10.1097/00004583-200001000-00014

52.

Shortt

A. L.

Barrett

P. M.

Fox

T. L.

(2001). Evaluating the FRIENDS program: A cognitive-behavioral group treatment for anxious children and their parents. Journal of Clinical Child Psychology, 30, 525-535. doi:10.1207/S15374424JCCP3004_09

53.

Silverman

W. K.

Albano

A. M.

(1996). The Anxiety Disorders Interview Schedule for DSM-IV-child and parent versions. San Antonio, TX: Graywind Publications.

54.

Sprague

Walker

H. M.

(2000). Early identification and intervention for youth with antisocial and violent behavior. Exceptional Children, 66, 367-379. doi:10.1177/001440290006600307

55.

Sprague

Walker

H. M.

Sowards

Van Bloem

C. V.

Eberhardt

Marshall

(2002). Sources of vulnerability to school violence: Systems-level assessment and strategies to improve safety and climate. In Shinn

M. R.

Walker

H. M.

Stoner

(Eds.), Interventions for academic and behavior problems II: Preventive and remedial approaches (pp. 295-314). Bethesda, MD: National Association of School Psychologists.

56.

Szigethy

Kenney

Carpenter

Hardy

D. M.

Fairclough

Bousvaros

. . . DeMaso

D. R.

(2007). Cognitive-behavioral therapy for adolescents with inflammatory bowel disease and subsyndromal depression. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 1290-1298. doi:10.1097/chi.0b013e3180f6341f

57.

Taras

H. L.

(2004). School-based mental health services. Pediatrics, 113, 1839-1845.

58.

Van Amerigen

Manicini

Farvolden

(2003). The impact of anxiety disorders on educational achievement. Journal of Anxiety Disorders, 17, 561-571. doi:10.1016/S0887-6185(02)00228-1

59.

Viera

A. J.

Garrett

J. M.

(2005). Understanding interobserver agreement: The kappa statistic. Family Medicine, 37, 360-363.

60.

Weissberg

R. P.

Durlak

J. A.

Domitrovich

C. E.

Gullotta

T. P.

(2015). Social and emotional learning: Past, present, and future. In Durlak

J. A.

Domitrovich

C. E.

Weissberg

R. P.

Gullotta

T. P.

(Eds.), Handbook of social and emotional learning: Research and practice (pp. 3-19). New York, NY: Guilford Press.

61.

Weist

M. D.

Youngstrom

E. A.

Stephan

Lever

Fowler

Taylor

. . . Hoagwood

(2014). Challenges and ideas from a research program on high-quality, evidence-based practice in school mental health. Journal of Clinical Child and Adolescent Psychology, 43, 244-255. doi:10.1080/15374416.2013.833097

62.

Weisz

J. R.

Southam-Gerow

M. A.

Gordis

E. B.

Connor-Smith

J. K.

Chu

B. C.

Langer

D. A.

. . . Weiss

(2009). Cognitive-behavioral therapy versus usual clinical care for youth depression: An initial test of transportability to community clinics and clinicians. Journal of Consulting and Clinical Psychology, 77, 383-396. doi:10.1037/a0013877

63.

Weisz

J. R.

Thurber

C. A.

Sweeney

Proffitt

V. D.

LeGagnoux

G. L.

(1997). Brief treatment of mild-to-moderate child depression using primary and secondary control enhancement training. Journal of Consulting and Clinical Psychology, 65, 703-707. doi:10.1037/0022-006X.65.4.703

64.

Wildeboer

Thijssen

van IJzendoorn

M. H.

van der Ende

Jaddoe

V. W. V.

Verhulst

F. C.

. . . Bakermans-Kranenburg

M. J.

(2015). Early childhood aggression trajectories: Associations with teacher-reported problem behaviour. International Journal of Behavioral Development, 39, 221-234. doi:10.1177/0165025414562239