The Effects of Social Skills Interventions for Students With EBD and ASD: A Single-Case Meta-Analysis

Abstract

Social skills interventions are critical for promoting social, emotional, and behavioral competence for students with or at risk of emotional and behavioral disorders (EBD) and autism spectrum disorders (ASD). This single-case meta-analysis examined the effects of social skills interventions (SSIs) for students with EBD and ASD. Effect sizes were calculated for 78 cases across 25 included studies using a nonparametric effect size, Baseline Corrected Tau. The overall weighted mean effect size of 0.54 suggested a moderate effect across the 25 studies. The overall weighted mean effects for studies reporting maintenance and generalization data were 0.68 and 0.37, respectively. Potential moderators examined (disability, intervention design, intervention delivery, methodological quality) were not significant. As such, they did not moderate the outcomes for participants. We conducted a post hoc analysis and hypothesized that between-study differences may be more meaningful than the similarities shared by participants in the same moderator groups. Implications are discussed on using SSIs to address the social, emotional, and behavioral challenges of students with or at risk of EBD and ASD.

Keywords

meta-analysis social skills challenging behavior autism emotional and behavioral disorders

The social, emotional, and behavioral challenges posed by students with emotional and behavioral disorders (EBD) and autism spectrum disorders (ASD) are a common issue in school-based settings (Forness, Freeman, Paparella, Kauffman, & Walker, 2012; H. M. Walker, Colvin, & Ramsey, 1995; S. Wang, Cui, & Parrila, 2011). Students with EBD exhibit externalizing problems and antisocial behaviors that include a conduct-related issues, failure to abide by rules, defiance of authority, conflicts with peers, and an avoidance of responsibility, as well as internalizing problems such as anxiety, depression, and withdrawal (Bradshaw, Schaeffer, Petras, & Ialongo, 2010; Forness et al., 2012; Sprague, Sugai, & Walker, 1998). Challenging social, emotional, and behavioral problems are characteristic of students with ASD as well, particularly those who are higher functioning (S. Wang et al., 2011). For example, it is not uncommon for individuals with ASD to experience social skills deficits, aggression, self-injurious behavior, and the inability to regulate or appropriately adjust their behaviors (e.g., McDougle, 2016). Although the etiology is different, these behavioral challenges can impede social, behavioral, and academic progress for both students with EBD and ASD.

Social Skills Interventions (SSIs)

SSIs are critical for promoting social, emotional, and behavioral competence for students with EBD and/or ASD. Social skills are critical to promoting social, emotional, and behavioral competence (Gresham, 2015). Gresham (2015) described social skills as being “academic enablers” and competing problem behaviors as “academic disablers” (p. 101). Moreover, social skills have been conceptualized as consisting of skill-based or performance-based deficits (Gresham, Sugai, & Horner, 2001; H. M. Walker, Ramsey, & Gresham, 2004). According to this social–behavioral model (Gresham, 1981, 1985, 2018), students have either not learned a particular social skill (namely, skill deficit), or they have learned the skill but do not perform it (namely, performance deficit). Conceptually, SSIs have developed over time from an intervention focus on discrete behavioral skills (e.g., eye contact) to now include social, cognitive, and emotional constructs and can be categorized into at least three areas: (a) social interactions, (b) prosocial behavior, and (c) social–cognitive skills (Gresham, 2015). Both group and single-case research designs have been used to evaluate SSIs for children and youth with EBD and/or ASD. Previous meta-analytic reviews on SSIs have included studies using single-case research designs for students with EBD (Mathur, Kavale, Quinn, Forness, & Rutherford, 1998) and students with ASD (e.g., Bellini, Peters, Benner, & Hopf, 2007; N. Schneider, Goldstein, & Parker, 2008; S. Wang et al., 2011; S. Wang, Parrila, & Cui, 2013). Others reviewed group designs that used SSIs for students with EBD (e.g., Ang & Hughes, 2002; Beelmann, Pfingsten, & Lösel, 1994; Cook et al., 2008; Quinn, Kavale, Mathur, Forness, & Rutherford, 1999). P. Wang and Spillane (2009) reviewed both single-case research (SCR) and group studies for students with ASD.

There are a number of narrative and quantitative reviews summarizing SSIs that collectively point to SSIs as being effective in promoting the acquisition and performance of social skills for students with or at risk of EBD (Cappadocia & Weiss, 2011; Flynn & Healy, 2012; Gillis & Butler, 2007; Gresham, 1981, 1985; Gresham & MacMillan, 1997; Maag, 2006; Reichow, Volkmar, & Cicchetti, 2008; White, Keonig, & Scahill, 2007). The meta-analyses of group design studies evaluating the effects of SSIs for students with or at risk of EBD have reported low to moderate effect sizes (Ang & Hughes, 2002; Beelmann et al., 1994; Cook et al., 2008; Quinn et al., 1999; B. H. Schneider, 1992). For example, B. H. Schneider (1992) reported an overall average effect size of r = .40 across 79 reviewed studies for participants that included children and youth described as “withdrawn,” “aggressive,” “at-risk,” and “unpopular with peers” (p. 368). In the meta-analysis by Beelmann et al. (1994), larger effect sizes were reported for problem-solving measures (g = .78), whereas smaller effect sizes were reported for behavior ratings (g = .26) of students with externalizing and internalizing behaviors, and those described as “at-risk” (p. 263). Later meta-analyses also reported a range of outcomes. Quinn et al. (1999) reported an overall effect size of d = .20 across 35 group design studies for students with EBD, whereas Ang and Hughes (2002) analyzed 38 studies on SSIs for antisocial youth with an overall effect size of ∆ = .62. Cook et al. (2008) synthesized the meta-analytic literature on SSIs for secondary students with or at risk of EBD. Their study included five meta-analyses and reported an overall weighted mean effect size of r = .32, indicating two thirds of students with or at risk of EBD (compared with one third of students in control groups) showed improvements following intervention.

Several reviews of SSIs have focused on students with ASD. Bellini et al. (2007) used percent of nonoverlapping data (PND; Scruggs, Mastropieri, & Casto, 1987) to analyze 55 single-case design studies researching SSIs for children with ASD resulting in a mean PND of 70%. P. Wang and Spillane (2009) reviewed 36 single-case and two-group design studies that included students with ASD. They found PNDs of 67.21% for social stories, 60.69% for peer-mediated, 84.25% for video modeling, and 100% for cognitive-behavioral training. N. Schneider et al. (2008) used the percentage of all nonoverlapping data points (PAND; Parker, Hagan-Burke, & Vannest, 2007) and the Phi statistic to calculate an overall effect size. An average weighted Phi of .71 was found for peer-mediated SSIs for students with ASD. Two single-case meta-analyses on SSIs for students with ASD used hierarchical linear modeling (HLM) to examine outcomes across multiple social skills studies (S. Wang et al., 2011; S. Wang et al., 2013). Both studies indicated that SSIs were effective in improving the social behavior of children with ASD, reporting γ = 1.27 (S. Wang et al., 2011) and γ = 1.40 (S. Wang et al., 2013).

In general, the previous reviews and meta-analysis point to greater effects for students with ASD than EBD. Mathur et al. (1998) was the only review that conducted a meta-analysis of single-case research that included both students with EBD and ASD. A total of 64 studies were included with 283 students with or at risk of EBD and students with ASD. The mean PND across all studies was 62% (SD = 33%).

Several meta-analyses identified differential effects for SSIs based on a number of variables including theoretical approach, intervention type, group composition, intervention strategy, implementation format, disability, and school level (Ang & Hughes, 2002; Bellini et al., 2007; Cook et al., 2008; Mathur et al., 1998; B. H. Schneider, 1992; P. Wang & Spillane, 2009). With regard to outcome measures, greater intervention effects were reported for promoting social interaction skills than communication skills (Mathur et al., 1998). Few reviews reported the effects of SSIs on maintenance or generalization. However, P. Wang and Spillane (2009) reported mean PND effect sizes for those single-case studies that included maintenance and generalization and reported an overall maintenance PND of 78.5% (range = 38%-100%), and an overall generalization of 80.95% (range = 40%-100%). In addition, Bellini et al. (2007) reported a range of maintenance effects (n = 25; PND M = 80%, range = 17%-100%) and lower generalization effects (n = 15; PND M = 53%, range = 17%-100%).

Single-Case Meta-Analysis

Single-case research relies on direct and systematic replication to build an evidence base for a particular practice or intervention (Horner et al., 2005). Meta-analysis procedures are increasingly being applied to single-case designs to empirically evaluate the external validity and generalizability of research using single-case designs (Kratochwill & Levin, 2014). An individual single-case design has strong internal validity but weak external validity (Horner et al., 2005), and meta-analytic procedures provide a well-established method of aggregating individual studies. However, there is no consensus about the best method for the data analysis of single-case studies. Several statistical methods (nonoverlap, regression based, etc.) are currently promoted for the quantitative analysis of single-case studies, each with unique strengths and limitations. One popular effect size statistic recommended for single-case meta-analysis is Tau-U, a modified form of Kendall’s rank order correlation Tau (Parker, Vannest, & Davis, 2011; Parker, Vannest, Davis, & Sauber, 2011; Vannest, Parker, Gonen, & Adiguzel, 2016). Tau-U has been adopted in a number of single-case meta-analyses (e.g., Bowman-Perrott et al., 2013; Dart, Collins, Klingbeil, & McKinley, 2014; Whalon, Conroy, Martinez, & Werch, 2015). However, a number of limitations have been noted by Tarlow (2017b) regarding the original Tau-U statistic. Notably, the baseline trend correction for the original Tau-U was found to produce inflated effect size estimates and “out-of-bounds” results, with Tau coefficients regularly falling outside of the −1 to +1 range. Tarlow introduced an improved rank correlation effect size statistic, Baseline Corrected Tau, which was designed to improve upon the Tau-U limitations.

Purpose and Research Questions

The purpose of this meta-analysis was to evaluate the overall effectiveness of single-case studies from 1998 to 2018 investigating SSIs for students with or at risk of EBD and ASD who display challenging social, emotional, or behavioral problems. Hutchins, Burke, Hatton, and Bowman-Perrott (2016) previously conducted a quality review extending the research conducted by Mathur et al. (1998) where the methodological quality of single-case social skills intervention studies from 1998 to 2014 was evaluated. The goal of the current study was to extend this prior quality review and focus on the effects of the SSIs included in Hutchins et al.’s (2016) study. Previous meta-analyses on SSIs have reported low to moderate effect sizes (Ang & Hughes, 2002; Cook et al., 2008; Mathur et al., 1998; Quinn et al., 1999). There has only been one single-case meta-analysis to date that specifically focused on SSIs that included both students with EBD and ASD (Mathur et al., 1998). The following research questions were examined:

Research Question 1: What is the overall effect of SSIs for students with or at risk of EBD and ASD?

Research Question 2: What are the effects of SSIs on maintenance and generalization?

Research Question 3: Are there differential effects for SSIs from studies included based on disability, intervention design, intervention delivery, or methodological quality?

Method

Study Identification

The initial pool of studies for this meta-analysis was developed from the quality review by Hutchins et al. (2016) of the single-case SSIs literature from 1998 to 2014. This data set was updated with an electronic search of psychology and educational databases (PsycINFO, ERIC, Academic Search Complete, and Education Full Text) to identify social skills intervention research from 1998 through 2018. Two sets of search terms were combined with the Boolean phrase AND. The first set of terms included behavioral disorders, emotional disorders, seriously emotionally disturbed, disruptive behavior, social behavior problems, antisocial behavior, autism, social behavior problems OR conduct disorders. The second set of terms included social skills training, social skills instruction OR social skills interventions. The search identified one additional article, resulting in a total of 24 articles published between 1998 and 2018 to be included in the current meta-analytic review (see Figure 1).

Figure 1.

Article selection flowchart.

Inclusion and Exclusion Criteria

A total of 25 studies across 24 articles were included in this meta-analysis. Studies were included if (a) the study was conducted in a school setting; (b) the participants were described as children or youth with or at risk of EBD or as ASD who exhibited challenging social and behavioral problems; (c) the intervention implemented was based on teaching social skills; (d) the primary outcome measures assessed school- or classroom-based social behaviors; (e) they used single-case design methodology; and (f) they were written in English, conducted in the United States, and published in a peer-reviewed journal between 1998 and 2018. Although often valuable for examining the “gray literature,” dissertations and unpublished manuscripts were excluded and only published studies were considered for review that had undergone a peer review process. Because the focus of this study was on school-based settings, studies were also excluded if they were conducted in clinical facilities or in the home environment, or if the dependent measures did not include school- or classroom-related behaviors, use a group design, or did not report on experimental findings (e.g., systematic literature reviews, editorials, commentaries, or practitioner guides).

Study Coding

Information was extracted for coding moderators. Potential moderator variables evaluated in the present study were (a) disability, (b) intervention design, (d) intervention delivery, and (e) methodological quality. The coding scheme from Hutchins et al. (2016) was used in the current meta-analysis. Disability was coded as identified with/at risk of EBD or identified with ASD. Intervention design refers to the individualization of the SSIs to the student. Interventions were considered individualized if they were created specifically for the student and matched social skills taught to the student’s social skill deficits (i.e., problem behavior). Studies implementing established social skills curricula which taught a variety of social skills were coded as general SSIs. Intervention delivery was coded as alone or combined. Studies in which social skills training was the only intervention implemented were considered stand-alone SSIs and were coded alone. Studies in which SSIs were combined with other strategies such as self-monitoring, cueing, group contingency, or other forms of reinforcement were considered combined SSIs. Overall methodological quality ratings were coded as yes if the study met What Works Clearinghouse (WWC) design standards with or without reservations (Kratochwill et al., 2013), and no if the study did not meet design standards. Additional details on the methodological quality review process can be found in Hutchins et al. (2016).

Data Extraction

Numerical values for each graphed data point were extracted to format graphed data into comma separated files and calculate effect sizes between phase contrasts. A digitizing computer software program was used to extract graphed data from the included articles (GetData, 2012). For each study, the figures containing graphed data for measures of outcome variables were cropped and uploaded into the software program. Once uploaded, and the scale of the X and Y axes were set, values for each data point were extracted and exported into an Excel spreadsheet and formatted for analysis. Values were rounded to the nearest whole number when necessary. Excel spreadsheets containing extracted data were then categorized by baseline and intervention phases. Data for maintenance and generalization phases were also categorized where applicable.

Reliability

Reliability for article selection, descriptive coding, methodological quality coding, and effect size calculations were assessed. Simple percent agreement and Cohen’s Kappa, a more conservative measure of reliability adjusting for chance agreement (Suen & Ary, 1989), were calculated for each area of reliability. Simple percent agreement was calculated by dividing the sum of agreements by the total number of agreements plus disagreements multiplied by 100. Cohen’s Kappa was calculated using the Vassar stats website (Lowry, 2001). Percent agreement above 80% and Cohen’s Kappa values above .60 are considered acceptable (Kratochwill et al., 2013). Percent agreement and Cohen’s Kappa for article selection, descriptive coding, and methodological quality were 90% and .81, 87% and .70, and 100% and 1.0, respectively.

Data Analyses

Baseline Corrected Tau was used for effect sizes and was calculated with the R statistical software package (R Core Team, 2017) using syntax by Tarlow (2017a). It is noted that an online calculator for Baseline Corrected Tau is also available and yields the same results as the R syntax (http://ktarlow.com/stats/tau; Tarlow, 2016). Random-effects models were used to aggregate the 78 participant effect size estimates, both within their respective 25 studies and for one overall weighted mean. Four subgroup effects were also analyzed as moderators with random-effects models. R syntax adopted from Borenstein, Hedges, Higgins, and Rothstein (2009) was used to aggregate the data (Tarlow & Brossart, 2018). Before aggregating effect size estimates, Tau values were transformed to z statistics and mean z values were transformed back to Tau for reporting and interpretation (Kendall, 1962; D. A. Walker, 2003). The signs (+/–) of effect sizes were reversed so that positive effects sizes indicated improvement, regardless of whether intervention outcome reflected increasing or decreasing behavior. The random-effects meta-analyses yielded two heterogeneity statistics, Q and I². Q tests the null hypothesis that between-participant variability is due only to sampling error. I² indicates the percentage of observed between-participant variance beyond what would be expected from sampling error. When Q is statistically significant, and/or when I² is large, there is reason to suspect participants are not homogeneous and may have been sampled from different populations. Subgroup (i.e., moderator) analyses tested the effect of (a) disability, (b) intervention design, (d) intervention delivery, and (e) methodological quality. Random-effects models with pooled variance (τ²) yielded Q^*between and R² statistics. The Q^*between statistic may be interpreted similarly to an analysis of variance (ANOVA) F test. Statistically significant Q^*between values indicate subgroup differences (i.e., moderator effects) are unlikely to be explained by sampling error. The R² statistic indicated the proportion of variance (τ²) explained by the moderator variable, analogous to the ANOVA η² (Borenstein et al., 2009). Intervention effect sizes and moderator analyses were all calculated using the first baseline-to-intervention phase contrast (i.e., A to B phase contrast). In cases where reversal designs (e.g., ABAB) were used, only the first A to B contrast was used to calculate an effect size. However, when data were available, effect sizes were calculated for maintenance (M) and generalization (G) probes with A to M phase contrasts and A to G phase contrasts, respectively. Ten studies with a total of 34 participants reported maintenance data, and three studies with a total of 11 participants reported generalization data. Maintenance and generalization effect sizes were aggregated within each study, and overall mean maintenance and generalization effects were estimated.

Results

Participant and Study Characteristics

Twenty-five studies across 24 articles published between 1998 and 2018 were analyzed. Studies included 78 participants with the majority of participants identified as male (85%, n = 66). Although race/ethnicity was not reported for 23 participants (29%), African American (32%, n = 25) and Caucasian (33%, n = 26) were the two ethnic groups with the greatest representation. All studies involved students with behavioral difficulties with 20 participants (26%) at risk of EBD, 28 participants (36%) identified with EBD, and 31 participants (38%) with ASD. The majority of students were educated in special education settings including specialized schools for students with disabilities (44%, n = 34), followed by general education settings (38%, n = 30). Fifteen of the participants (19%) were educated in both special education and general education settings, and more than half (53%, n = 42) of the interventions implemented were individualized to the participant. Multiple baseline designs were the most commonly used experimental design (68%, n = 17), followed by reversal/variation designs (16%, n = 4), and mixed designs (16%, n = 4). Finally, 11 studies (41% of participants; n = 32) evaluated the effect of SSIs on the increase of social skill behaviors, five studies (17% of participants, n = 13) evaluated the effect of SSIs on the decrease of problem behaviors, and nine studies (42% of participants, n = 33) collected data to evaluate the simultaneous increase of social skill and decrease of problem behavior.

Mean Effect Size of Social Skills Interventions

Random-effects means were first calculated to aggregate the participant effect size estimates within each of the 25 studies (six studies had only one participant). Mean effect sizes ranged from 0.22 to 0.75. The study-level mean effects were statistically significant for all but three studies. The random-effects mean for all 78 participants was Baseline Corrected Tau = 0.541, 95% confidence interval (CI) = [0.486, 0.591], p < .001. This is equivalent to a Cohen’s d effect size of 2.240. The weighted mean effects are summarized in Table 1. Heterogeneity statistics for the overall mean effect were statistically significant (Q = 453.93, p < .001) and indicated a large amount of variance among participants’ intervention effect sizes that were not explained by sampling error (I² = 0.83). It was therefore expected that some moderator variable(s) accounted for this heterogeneity. Maintenance and generalization effect sizes are summarized in Table 2. The weighted mean effect sizes for all studies with maintenance (n = 10) and generalization (n = 3) data were statistically significant. The overall mean effect size for maintenance data (n = 34 participants) was Baseline Corrected Tau = 0.679, 95% CI = [0.630, 0.722], p < .001. The overall mean effect size for generalization data (n = 11 participants) was Baseline Corrected Tau = 0.369, 95% CI = [0.210, 0.506], p < .001.

Table 1.

Within-Study Random-Effects Meta-Analyses.

Study	Cases	BC-Tau	95% CI	Q	I ²
Bardon, Dona, and Symons (2008)	3	0.686***	[0.590, 0.762]	0.303	0.000
Blake, Wang, Cartledge, and Gardner (2000)	3	0.293**	[0.113, 0.451]	3.121	0.359
Blake et al. (2000)	6	0.721***	[0.677, 0.758]	2.163	0.000
Blood, Johnson, Ridenour, Simmons, and Crouch (2011)	1	0.714***	[0.454, 0.855]	—	—
Bock (2007a)	1	0.588	[–0.529, 0.945]	—	—
Bock (2007b)	4	0.611***	[0.496, 0.703]	0.050	0.000
Campbell and Tincani (2011)	3	0.653***	[0.536, 0.742]	3.087	0.352
Chan and O’Reilly (2008)	2	0.624**	[0.274, 0.817]	3.443	0.710
Hagiwara and Myles (1999)	3	0.490	[–0.188, 0.832]	27.307***	0.927
Hagopian, Kuhn, and Strother (2009)	1	0.730***	[0.460, 0.870]	—	—
Hansen and Lignugaris-Kraft (2005)	8	0.239	[–0.068, 0.498]	37.023***	0.811
Herring and Northup (1998)	1	0.713*	[0.213, 0.906]	—	—
Hune and Nelson (2002)	4	0.344***	[0.229, 0.449]	2.007	0.000
Keeling, Myles, Gagnon, and Simpson (2003)	1	0.513***	[0.269, 0.688]	—	—
Kelly and Shogren (2014)	4	0.696***	[0.622, 0.757]	0.956	0.000
Kuoch and Mirenda (2003)	3	0.733***	[0.508, 0.858]	7.017*	0.715
Lo, Loe, and Cartledge (2002)	5	0.318**	[0.125, 0.484]	21.914***	0.817
D. N. Miller and Cole (1998)	1	0.683***	[0.517, 0.795]	—	—
M. A. Miller, Fenty, Scott, and Park (2011)	3	0.338***	[0.199, 0.461]	1.191	0.000
Ozdemir (2008)	3	0.486***	[0.357, 0.594]	3.131	0.361
Presley and Hughes (2000)	4	0.753***	[0.666, 0.819]	3.359	0.107
Ross and Sabey (2015)	4	0.514***	[0.373, 0.628]	3.509	0.145
Scattone, Tingstrom, and Wilczynski (2006)	3	0.526**	[0.152, 0.756]	21.332***	0.906
Schneider and Goldstein (2010)	3	0.409***	[0.274, 0.526]	3.229	0.381
Simpson, Langone, and Ayres (2004)	4	0.460	[–0.173, 0.804]	60.469***	0.950
All Studies	78	0.541***	[0.486, 0.591]	453.925***	0.830

Note. BC = baseline corrected; CI = confidence interval.

p < .05. **p < .01. ***p < .001.

Table 2.

Maintenance and Generalization Effect Sizes.

Study	Cases	Maintenance				Generalization
Study	Cases	BC-Tau	95% CI	Q	I ²	BC-Tau	95% CI	Q	I ²
Blake, Wang, Cartledge, and Gardner (2000)	3	0.506***	[0.316, 0.652]	0.019	0.000
Blake et al. (2000)	6	0.698***	[0.581, 0.784]	16.218**	0.692
Bock (2007b)	4	0.780***	[0.664, 0.857]	0.141	0.000
Campbell and Tincani (2011)	3	0.612**	[0.277, 0.804]	16.637***	0.880
Chan and O’Reilly (2008)	2	0.818***	[0.642, 0.908]	0.122	0.000
Herring and Northup (1998)	1	0.775**	[0.435, 0.915]	—	—
Hune and Nelson (2002)	4	0.752***	[0.575, 0.858]	0.305	0.000	0.307***	[0.191, 0.413]	2.170	0.000
Kelly and Shogren (2014)	4	0.681***	[0.568, 0.767]	1.031	0.000
Lo, Loe, and Cartledge (2002)	5					0.313*	[0.030, 0.542]	50.271***	0.920
M. A. Miller, Fenty, Scott, and Park (2011)	2					0.664***	[0.485, 0.786]	0.925	0.000
Ozdemir (2008)	3	0.529***	[0.364, 0.657]	1.806	0.000
Ross and Sabey (2015)	4	0.734***	[0.600, 0.825]	0.446	0.000
All Studies	34; 11	0.679***	[0.630, 0.722]	57.802**	0.429	0.369***	[0.210, 0.506]	64.923***	0.846

Note. BC = baseline corrected; CI = confidence interval.

p < .05. **p < .01. ***p < .001.

Moderator Analyses

Disability

Two disability categories were tested for between-group differences: students with or at risk of EBD (n = 46) and students with ASD (n = 32). There was no statistically significant difference between the two groups (Q^*between = 0.418, p = .441). Disability type did not account for any proportion of variance among the participants’ effect sizes, R² = .000 (see Table 3).

Table 3.

Random-Effects Moderator Analyses.

Moderator	Cases	BC-Tau	95% CI	Q ^*between	R ²
Disability	78			0.595	.000
At-risk/EBD	46	0.524***	[0.449, 0.590]
ASD	32	0.565***	[0.481, 0.638]
Intervention design	78			0.269	.000
Individualized	41	0.554***	[0.478, 0.621]
General	37	0.527***	[0.444, 0.599]
Intervention delivery	78			2.694	.000
Social skills	48	0.574***	[0.508, 0.632]
Combined	30	0.481***	[0.380, 0.569]
Quality standards met	78			0.479	.000
Yes	59	0.559***	[0.478, 0.629]
No	19	0.526***	[0.449, 0.595]
Study (post hoc)	78			62.029***	.429

Note. BC = baseline corrected; CI = confidence interval; EBD = emotional and behavioral disorders; ASD = autism spectrum disorders.

p < .05. **p < .01. ***p < .001.

Intervention design

Two intervention design categories were tested for between-group differences: individualized (n = 41) and general (n = 37). There was no statistically significant difference between the two groups (Q^*between = 0.177, p = .604). Intervention design did not account for any proportion of variance among the participants’ effect sizes, R² = .000.

Intervention delivery

Two intervention delivery categories were tested for between-group differences: social skills (n = 48) and combined (n = 30). There was no statistically significant difference between the two groups (Q^*between = 2.660, p = .101). Intervention delivery did not account for any proportion of variance among the participants’ effect sizes, R² = .000.

Methodological quality

Two methodological quality categories were tested for between-group differences: participants from studies that met quality standards (n = 59) and participants from studies that did not meet quality design standards (n = 19). There was no statistically significant difference between the two groups (Q^*between = 0.220, p = .489). Methodological quality did not account for any proportion of variance among the participants’ effect sizes, R² = .000.

Between-study differences (post hoc)

After considering the poor predictive power of the four planned moderator analyses, we decided to examine the effect of study membership on participant effect size (i.e., how much variation in effect size is accounted for by characteristics of the studies themselves). It was hypothesized that between-study differences may be more meaningful than the similarities shared by participants in the same moderator groups. Indeed, there were statistically significant differences between study mean effect sizes (Q^*between = 68.410, p < .001). Study membership accounted for a considerable amount of variance among participants’ effect sizes, R² = .472.

Discussion

This study focused on examining the social skills intervention literature for students with or at risk of EBD, and students identified with ASD. This work extended previous research on SSIs from Mathur et al. (1998) and the quality review by Hutchins et al. (2016). This study addressed three main questions. First, the primary question focused on examining the overall effect of SSIs for students with or at risk of EBD and ASD. This review found a moderate overall effect across the SSIs. This finding is consistent with prior meta-analytic reviews of social skills intervention literature (e.g., Cook et al., 2008; Gresham, 2015). The effect size ranges reported in this current review ranged from relatively weak (e.g., Hansen & Lignugaris-Kraft, 2005) to moderately strong (e.g., Presley & Hughes, 2000). As illustrated in Table 1, when evaluating the social skills intervention evidence base, careful attention should be given to the specific intervention being studied.

The second question focused on the effects of SSIs on the maintenance and generalization of social skills. The finding that few social skills intervention studies collected maintenance and generalization data are consistent with previous reviews of social skills training including students with or at risk of EBD (Maag, 2006). For those studies that included maintenance and generalization data, the weighted mean effect sizes were 0.68 and 0.37. Similar to previous reports, greater effects were found for maintenance when compared with generalization of social skills. These findings suggest that some studies do have effects that maintain over time. However, generalization effects remain underexamined and problematic. Maintenance of social skills reported by Mathur et al. (1998) and Bellini et al. (2007) was greater than that reported in the current study. Generalization measures for students with ASD were found to be smaller in Bellini et al. (2007) and larger in P. Wang and Spillane (2009) than those obtained in the current study. Smaller generalization effects were reported in Mathur et al. (1998) than in this meta-analysis.

The third research question focused on the differential effects for SSIs. Previous meta-analyses have identified differential effects for SSIs based on theoretical approach, intervention type, group composition, intervention strategy, implementation format, disability, and school level. Variation effects were not found for the four moderator variables tested: disability, intervention design, intervention delivery, and methodological quality. A post hoc analysis was conducted because the moderator variables examined (e.g., disability, study quality) did not explain the observed variance, as indicated by their lack of statistical significance. As such, there may have been study-level differences that influenced the findings of this meta-analysis. In speculating reasons for the nonsignificance of the factors, it may be that factors not examined in this review (e.g., experimenter effects, type of behavior, how SSIs were delivered, how participants were sampled, and treatment fidelity) may account for the unexplained variance revealed by the post hoc analysis.

Limitations and Future Research

The following limitations and future research issues should be considered. First, Mathur et al. (1998) reported greater effects for students who were at risk of EBD than students with ASD or who were identified as delinquents. The current meta-analysis did not separate students formally identified with EBD from more “garden variety” students at-behavioral risk. Often times, there was not sufficient information to disaggregate the characteristics of the participants either (e.g., externalizing vs. internalizing). Those with formal diagnostic identifications may exhibit more problematic behavior. As the quality of reporting standards improves in the field, additional research should be conducted to disaggregate the participant characteristics of students included in SSIs. Second, differences in the effects of SSIs based on skill deficits versus performance deficits were not assessed. In part, because the information needed for this determination is not reported in the studies reviewed. Future research should examine the differential effects of social skills interventions on skill deficits or performance deficits for students with or at risk of EBD and ASD in future studies. Third, social skills and the construct of social competence encompass a wide range of skills and behaviors both in and out of school. Future research is needed to examine transfer (generalization) effects outside of school or classroom-based behaviors. Finally, participant outcomes (e.g., type of behavior such as aggression, initiating social interactions) and treatment fidelity should be examined as potential moderators in future meta-analyses. Examination of these moderators may help additional information on aspects of SSIs that are effective for children and youth with EBD and ASD.

Gresham (1998) posed the question of whether the field should “raze, remodel, or rebuild” the social skills literature. Moreover, Gresham (2015) recently noted that “about 65% of students with EBD will improve when given social skills interventions” (p. 100). Noting that much progress has been made in the research literature on SSIs, Gresham indicated that the level of improvement reported from the meta-analytic literature indicates that SSIs are not as robust as one would expect. This range of effects found in this meta-analysis is similar to previous reviews found in the social skills literature. There is clearly a range of responsiveness to SSIs. Those studies that report maintenance and generalization confirm the modest effects of transfer of social skills to other conditions. From an idiographic perspective, more attention should be given specifically to those studies that have yielded a strong response to intervention. The unique characteristics of the interventions in these studies should be attended to for the purposes of replication and implementation in applied settings. In moving forward, our hope is this current meta-analysis will assist researchers in “remodeling” the social skills intervention literature.

Footnotes

Appendix

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Lisa Bowman-Perrott

Author Biographies

Nancy S. Hutchins, PhD, serves as clinical professor and director of Diversity and Inclusion for the Mays Business School at Texas A&M University. She is a trained mediator, and facilitates workshops in the areas of diversity, equity, inclusion, campus climate, and strategic conflict engagement. Dr. Hutchins has worked as an educator in public schools with a focus on special education, behavior intervention, positive behavior support, and classroom management.

Mack D. Burke, PhD, is an associate professor in the Department of Educational Psychology, Special Education Division, at Texas A&M University. He is affiliated with the Center on Disability and Development, and conducts research through the Behavior and Academic Support Collaborative. Dr. Burke’s areas of research interest and scholarship include emotional and behavioral disorders, positive behavioral supports, social and emotional learning, and evidence-base social and behavioral practices.

Lisa Bowman-Perrott, PhD, is an associate professor in the Department of Educational Psychology, Special Education Division, at Texas A&M University. Her research focuses on academic and behavioral interventions for students with or at-risk for emotional and behavioral disorders, as well as effective interventions for English language learners. Dr. Bowman-Perrott examines the efficacy of widely used interventions through meta-analysis of single-case research as an extension of her work implementing interventions in schools.

Kevin R. Tarlow, PhD, is an assistant professor of Psychology at Hampton University. He studies psychotherapy, trauma, telepsychology, and mental health disparities. Dr. Tarlow’s research also includes the development of new single-case research methods for clinical scientists and student researchers.

Heather Hatton, PhD, is an assistant research professor in the Department of Special Education at the University of Missouri. Her research interests are in emotional and behavioral disorders, positive behavioral support, evidence-based interventions, and meta-analysis. Dr. Hatton also examines models for pre-service and in-service teacher training.

References

Ang

R. P.

Hughes

J. N.

(2002). Differential benefits of skills training with antisocial youth based on group composition: A meta-analytic investigation. School Psychology Review, 31, 164-185.

Beelmann

Pfingsten

Lösel

(1994). Effects of training social competence in children: A meta-analysis of recent evaluation studies. Journal of Clinical Child Psychology, 23, 260-271.

Bellini

Peters

J. K.

Benner

Hopf

(2007). A meta-analysis of school-based social skills interventions for children with autism spectrum disorders. Remedial and Special Education, 28, 153-162.

Borenstein

Hedges

L. V.

Higgins

J. P. T.

Rothstein

H. R.

(2009). Introduction to meta-analysis. West Sussex, UK: John Wiley.

Bowman-Perrott

Davis

Vannest

Williams

Greenwood

Parker

(2013). Academic benefits of peer tutoring: A meta-analytic review of single-case research. School Psychology Review, 42, 39-55.

Bradshaw

C. P.

Schaeffer

C. M.

Petras

Ialongo

(2010). Predicting negative life outcomes from early aggressive-disruptive behavior trajectories: Gender differences in maladaptation across life domains. Journal of Youth and Adolescence, 39, 953-966. doi:10.1007/s10964-009-9442-8

Cappadocia

M. C.

Weiss

J. A.

(2011). Review of social skills training groups for youth with Asperger Syndrome and high functioning autism. Research in Autism Spectrum Disorders, 5, 70-78.

Cook

C. R.

Gresham

F. M.

Kern

Barreras

R. B.

Thornton

Crews

S. D.

(2008). Social skills training for secondary students with emotional and/or behavioral disorders: A review and analysis of the meta-analytic literature. Journal of Emotional and Behavioral Disorders, 16, 131-144.

Dart

E. H.

Collins

T. A.

Klingbeil

D. A.

McKinley

L. E.

(2014). Peer management interventions: A meta-analytic review of single-case research. School Psychology Review, 43, 367-384.

10.

Flynn

Healy

(2012). A review of treatments for deficits in social skills and self-help skills in autism spectrum disorder. Research in Autism Spectrum Disorders, 6, 431-441.

11.

Forness

S. R.

Freeman

S. F. N.

Paparella

Kauffman

J. M.

Walker

H. M.

(2012). Special education implications of point and cumulative prevalence for children with emotional or behavioral disorders. Journal of Emotional and Behavioral Disorders, 20, 4-18. doi:10.1177/1063426611401624

12.

GetData. (2012). GetData Graph Digitizer [Computer software]. Retrieved from http://getdata-graph-digitizer.com/index.php

13.

Gillis

J. M.

Butler

R. C.

(2007). Social skills interventions for preschoolers with autism spectrum disorder: A description of single-subject design studies. Journal of Early and Intensive Behavior Intervention, 4, 532-547.

14.

Gresham

F. M.

(1981). Social skills training with handicapped children: A review. Review of Educational Research, 51, 139-176.

15.

Gresham

F. M.

(1985). Utility of cognitive-behavioral procedures for social skills training with children: A critical review. Journal of Abnormal Child Psychology, 13, 411-423.

16.

Gresham

F. M.

(1998). Social skills training: Should we raze, remodel, or rebuild? Behavior Disorders, 24(1), 19-25.

17.

Gresham

F. M.

(2015). Disruptive behavior disorders: Evidence-based practice for assessment and intervention. New York, NY: Guilford Press.

18.

Gresham

F. M.

(2018). Effective interventions for social-emotional learning. New York, NY: Guilford Press.

19.

Gresham

F. M.

MacMillan

D. L.

(1997). Social competence and affective characteristics of students with mild disabilities. Review of Educational Research, 67, 377-415.

20.

Gresham

F. M.

Sugai

Horner

R. H.

(2001). Interpreting outcomes of social skills training for students with high-incidence disabilities. Exceptional Children, 67, 331-344.

21.

Hansen

S. D.

Lignugaris-Kraft

(2005). Effects of a dependent group contingency on the verbal interactions of middle school students with emotional disturbance. Behavioral Disorders, 30, 170-184.

22.

Horner

R. H.

Carr

E. G.

Halle

McGee

Odom

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165-179.

23.

Hutchins

Burke

M. D.

Hatton

Bowman-Perrott

(2016). Social skills interventions for students with challenging behavior: Evaluating the quality of the evidence base. Remedial and Special Education, 38, 13-27. doi:10.1177/0741932516646080

24.

Kendall

M. G.

(1962). Rank correlation methods (3rd ed.). New York, NY: Hafner.

25.

Kratochwill

T. R.

Hitchcock

J. H.

Horner

R. H.

Levin

J. R.

Odom

S. L.

Rindskopf

D. M.

Shadish

W. R.

(2013). Single-case intervention research design standards. Remedial and Special Education, 34, 26-38.

26.

Kratochwill

T. R.

Levin

J. R.

(2014). Meta- and statistical analysis of single-case intervention research data: Quantitative gifts and a wish list. Journal of School Psychology, 52, 231-235.

27.

Lowry

(2001). VassarStats: Website for statistical computation. Retrieved from http://vassarstats.net/

28.

Maag

J. W.

(2006). Social skills training for students with emotional and behavioral disorders: A review of reviews. Behavioral Disorders, 32, 4-17.

29.

Mathur

S. R.

Kavale

K. A.

Quinn

M. M.

Forness

S. R.

Rutherford

R. B.

(1998). Social skills interventions with students with emotional and behavioral problems: A quantitative synthesis of single-subject research. Behavioral Disorders, 23, 193-201.

30.

McDougle

C. J.

(2016). Autism spectrum disorders (1st ed.). New York, NY: Oxford University Press.

31.

Parker

R. I.

Hagan-Burke

Vannest

(2007). Percentage of All Non-Overlapping Data (PAND): An alternative to PND. The Journal of Special Education, 40, 194-204.

32.

Parker

R. I.

Vannest

K. J.

Davis

J. L.

(2011). Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification, 35, 303-322.

33.

Parker

R. I.

Vannest

K. J.

Davis

J. L.

Sauber

S. B.

(2011). Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy, 42, 284-299.

34.

Presley

J. A.

Hughes

(2000). Peers as teachers of anger management to high school students with behavioral disorders. Behavioral Disorders, 25, 114-130.

35.

Quinn

M. M.

Kavale

K. A.

Mathur

S. R.

Forness

S. R.

Rutherford

R. B.

(1999). A meta-analysis of social skill interventions for students with emotional or behavioral disorders. Journal of Emotional and Behavioral Disorders, 7, 54-64.

36.

R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Available from https://www.R-project.org

37.

Reichow

Volkmar

F. R.

Cicchetti

D. V.

(2008). Development of the evaluative method for evaluating and determining evidence-based practices in autism. Journal of Autism and Developmental Disorders, 38, 1311-1319.

38.

Schneider

B. H.

(1992). Didactic methods for enhancing children’s peer relations: A quantitative review. Clinical Psychology Review, 12, 363-382.

39.

Schneider

Goldstein

Parker

(2008). Social skills interventions for children with autism: A meta-analytic application of Percentage of All Non-Overlapping Data (PAND). Evidence-Based Communication Assessment and Intervention, 2, 152-162.

40.

Scruggs

T. E.

Mastropieri

M. A.

Casto

(1987). The quantitative synthesis of single subject research: Methodology and validation. Remedial and Special Education, 8(2), 24-33.

41.

Sprague

Sugai

Walker

(1998). Antisocial behavior in schools. In Watson

T. S.

Gresham

F. M.

(Eds.), Handbook of child behavior therapy (pp. 451-474). New York, NY: Springer.

42.

Suen

H. K.

Ary

(1989). Analyzing quantitative behavioral observation data. Mahwah, NJ: Lawrence Erlbaum.

43.

Tarlow

K. R.

(2016). Baseline Corrected Tau Calculator. Retrieved from http://www.ktarlow.com/stats/tau

44.

Tarlow

K. R.

(2017a, June). Baseline Corrected Tau for single-case research (R code). Retrieved from http://ktarlow.com/stats/

45.

Tarlow

K. R.

(2017b). An improved rank correlation effect size statistic for single-case designs: Baseline Corrected Tau. Behavior Modification, 41, 427-467.

46.

Tarlow

K. R.

Brossart

D. F.

(2018). A comprehensive method of single-case data analysis: Interrupted Time-Series Simulation (ITSSIM). School Psychology Quarterly, 33, 590-603. doi:10.1037/spq0000273

47.

Vannest

K. J.

Parker

R. I.

Gonen

Adiguzel

(2016). Single Case Research: Web based calculators for SCR analysis (Version 2.0). College Station: Texas A&M University. Retrieved from http://singlecaseresearch.org

48.

Walker

D. A.

(2003). JMASM9: Converting Kendall’s Tau for correlational or meta-analytic analyses. Journal of Modern Applied Statistical Methods, 2, 525-530.

49.

Walker

H. M.

Colvin

Ramsey

(1995). Antisocial behavior in school: Strategies and best practices. Pacific Grove, CA: Brooks/Cole.

50.

Walker

H. M.

Ramsey

Gresham

F. M.

(2004). Antisocial behavior in school: Evidence-based practices. Belmont, CA: Wadsworth.

51.

Wang

Spillane

(2009). Evidence-based social skills interventions for children with autism: A meta-analysis. Education and Training in Developmental Disabilities, 44, 318-342.

52.

Wang

Cui

Parrila

(2011). Examining the effectiveness of peer-mediated and video-modeling social skills interventions for children with autism spectrum disorders: A meta-analysis in single-case research using HLM. Research in Autism Spectrum Disorders, 5, 562-569.

53.

Wang

Parrila

Cui

(2013). Meta-analysis of social skills interventions of single-case research for individuals with autism spectrum disorders: Results from three-level HLM. Journal of Autism and Developmental Disorders, 43, 1701-1716.

54.

Whalon

K. J.

Conroy

M. A.

Martinez

J. R.

Werch

B. L.

(2015). School-based peer-related social competence interventions for children with autism spectrum disorder: A meta-analysis and descriptive review of single case research design studies. Journal of Autism and Developmental Disorders, 45, 1513-1531.

55.

White

S. W.

Keonig

Scahill

(2007). Social skills development in children with autism spectrum disorders: A review of the intervention research. Journal of Autism and Developmental Disorders, 37, 1858-1868.