The Effects of School-Based Social Information Processing Interventions on Aggressive Behavior,Part I: Universal Programs

Social Information Processing Programs

Some of the earliest programs designed to target social information processing deficits were the social problem solving programs developed in the 1970s (e.g., D’Zurilla & Goldfried, 1971; Shure & Spivack, 1972; 1979). The programs were designed to improve social behavior by teaching cognitively-based problem solving skills. For example, the I Can Problem Solve program (Shure, 1992) involves teaching participants a series of problem solving skills that includes perspective taking, alternative response generation, consequential or means-ends thinking, and causal thinking.

In subsequent years, many variations on this theme have been developed (Coleman, Wheeler, & Webber, 1993). For instance, attribution retraining programs, such as Hudley's BrainPower (Hudley, 1991) program, focus on the early stages of the social information processing model and teach children to detect intentionality accurately, to make nonhostile attributions when social encounters are ambiguous, and to generate appropriate behavioral responses to ambiguous negative situations. Lochman's Anger Coping Program (Lochman, Lampron, Burch, & Curry, 1985) includes a module that helps children learn cognitive goal-setting skills, a social problem solving component, and a module on anger control (which focuses on the emotional aspects of social information processing).

While these are examples of some typical social information processing programs, a conceptual definition of the programs in general is necessary for conducting the proposed systematic review. This definition aids in identifying candidate programs for the review and in coding the particular treatment components of each program. For purposes of this review, therefore, a social information processing program is one with the following distinct characteristics:

The program involves training in one or more of the social information processing steps: (1) encoding situational and internal cues, (2) interpreting the cues, (3) selecting or clarifying a goal, (4) generating or accessing possible responses to meet goals, (5) choosing a response, (6) and enacting the behavior.

What Social Information Processing Programs are Not

Another set of popular programming strategies also focuses on social competence. These programs, often called by the generic term behavioral social skills programs, are distinct from the social information processing programs that are the focus of this review. Behavioral social skills curricula focus primarily on the social (or antisocial) behaviors themselves, rather than the underlying cognitive thought processes. These programs generally teach specific skills such as making eye contact, smiling in context, giving compliments, communication skills, group entry skills, assertiveness, and the like (see Goldstein & Pentz, 1984 for a review).

In addition, there are other cognitively-oriented programs that do not specifically target social information processing. For example, programs for children with attention or activity level difficulties often involve teaching skills for cognitive impulse control. Similarly, there are cognitively oriented programs for dealing with stress, divorce, depression, etc. (e.g., Alpert-Gillis, Pedro-Carroll, & Cowen, 1987; Lewinsohn, Clarke, Hops, & Andrews, 1990). While these other cognitive programs focus on thinking skills, they do not have the focus on interpersonal relationships that is characteristic of social information processing programs.

Universal Program Delivery

Programs that address social information processing difficulties tend to be structured and have detailed lesson plans, which makes them attractive to schools. These programs are easily delivered by teachers or school psychologists and can be used in different formats (group or individual) and settings (classrooms or out-of-class school facilities). They are common in school settings, but are also used in a variety of settings outside of school. Our original intent for this systematic review was to include all research studies of social information processing programs delivered in school settings. As we began collecting the literature on social information processing programs, however, we discovered that the programs were delivered using two complementary but distinct approaches to prevention—universal and selected or indicated prevention (Mrazek & Haggerty, 1994). Universal prevention interventions are delivered in classroom settings to an entire classroom; children are not selected individually for treatment but, rather, receive the program simply because they are in a program classroom. Selected or indicated prevention interventions target children who are exhibiting risk factors (such as anger or hyperactivity) or children who are already showing signs of disorder (i.e., behavior problems). In addition to the differences in the subject samples in universal versus selected or indicated programs, we noticed that there were other important differences between the two prevention approaches, most notably in study procedure and method. Nearly all of the studies of programs delivered to selected/indicated students used random assignment of individual subjects to produce treatment and comparison groups, while only a few studies of universal programs did so. Based on these differences in study and subject characteristics, and the possibility that these factors might be associated with study outcomes, we elected to conduct two separate systematic reviews. The review presented here focuses solely on universal social information processing programs delivered in school settings. A second review focuses on school-based selected/indicated social information processing programs (Wilson & Lipsey, forthcoming).

Prior Research on Social Skills Interventions

Several recent narrative reviews of social skills interventions are available (Clayton, Ballif-Spanvill, & Hunsaker, 2001; Howard, Flora, & Griffin, 1999; Moote, Smyth, & Wodarski, 1999; Nangle, Erdley, Carpenter, & Newman, 2002) but these cover social skills interventions quite broadly by including training in social skills, friendship making skills, communication skills, and other such social behaviors without an explicit focus on altering cognitive and social information processing deficits. A few narrative reviews of programs specific to social information processing have also been conducted (e.g., Coleman, Wheeler, & Webber, 1993; Pellegrini & Urbain, 1985), but these are somewhat dated and include programs delivered to clinical populations in non-school settings. Several meta-analysis of social skills interventions are available (e.g., Beelmann, Pfingsten, & Lösel, 1994; Denham & Almeida, 1987; Lösel & Beelmann, 2003; Quinn, Kavale, Mathur, Rutherford, & Forness, 1999; Schneider & Byrne, 1985). Like the narrative reviews, but with the exception of Denham and Almeida (1987), these syntheses focus on social competence programs in general and thus include both social information processing and behavioral social skills programs. The Denham and Almeida meta-analysis focuses exclusively on interpersonal cognitive problem solving interventions, but is 15 years old and does not include international studies.

Criteria for Including Studies in the Review

Interventions. Interventions were required to meet three criteria that define social information processing programs. Although other treatment components were allowed to be present (e.g., behavioral social skills training, parenting skills training), the social information processing component had to be the clear focus of the program. The definitional criteria were as follows:

Training was provided on one or more of the social information processing steps: (1) encoding situational and internal cues, (2) interpretation of cues, (3) selecting or clarifying a goal, (4) generating or accessing possible responses, (5) choosing a response, (6) and behavioral enactment.

Settings and Subjects. The interventions were delivered to school-aged children (K-12 or equivalent ages in international settings) in regular school settings during school hours. Eligible programs had to be delivered universally to all children in a school, classroom, or group of schools or classrooms. Special education classrooms and alternative schools were considered eligible school settings, although classrooms in residential facilities (e.g., psychiatric hospitals) were not. After-school programs were not eligible. Any qualifying school in any region or country was eligible.

Outcomes. The study reported intervention effects for at least one outcome variable, measured on children, representing aggressive behavior, broadly defined to include violence, aggression, fighting, person crimes, disruptive behavior problems, acting out, conduct disorder, externalizing problems, and so forth.

Study Design. Only studies using a control group design were eligible. The intervention and control groups could be randomly or nonrandomly assigned but, if nonrandom, needed to be matched or provide evidence of initial equivalence on key demographic variables and/or pretests. Control groups could represent placebo, wait-list, no treatment, or “treatment as usual” conditions. Studies without control or comparison groups were not eligible. This included one-group pretest-posttest studies and studies in which a treatment condition was compared to another treatment condition.

Search Strategy for Identification of Studies

An attempt was made to identify and retrieve the entire population of empirical studies that met the eligibility criteria specified above, including both published and unpublished studies. Several sources were used to identify potentially eligible research reports. First, a large database compiled at the Center for Evaluation Research and Methodology with NIMH grant funding (Lipsey, PI) was searched for eligible interventions. That database includes studies of early intervention programs targeting a wide range of risk factors for antisocial behavior and includes many school-based programs.

In addition, a comprehensive search of bibliographic databases, including Psychological Abstracts, Dissertation Abstracts International, ERIC (Educational Resources Information Center), the Campbell and Cochrane Collaboration trials registers, U.S. Government Printing Office publications, National Criminal Justice Reference Service, and MedLine was conducted, particularly to find recent studies not included in the existing database.

The search strategy utilized three categories of search terms2 derived from the controlled vocabulary used to index articles for each specific database. Wildcard characters were used to identify variants of words (e.g., delinquen* to locate delinquent, delinquents, and delinquency). Within each category described below, search terms are connected by the OR operator; the results for each of the three categories were combined with the AND operator:

The population of interest: child, adolescent, childhood (MEDLINE); children, adolescents, preadolescents, students (ERIC); children, adolescents, students (PsychInfo), and children, adolescents, students (Sociological Abstracts);

Third, the bibliographies of previous meta-analyses and literature reviews (e.g., Denham & Almeida, 1987; Durlak, 1997) were inspected for studies that met the eligibility criteria. In addition, the bibliographies of retrieved studies were themselves examined for potentially eligible research reports. Finally, follow-up searches on the first and second authors of all eligible studies and cited reference searches of eligible articles in the Social Sciences Citation Index were conducted. We have found this forward searching technique to be more efficient and fruitful than conducting hand searches of journal issues.

Studies identified through our searches were retrieved from the library, obtained via interlibrary loan, or requested directly from the author(s). In addition, we attempted to locate and include any eligible studies published in non-English language sources that were missed by the search strategies described above.

Selection of Studies

Abstracts of the studies found through the search procedures were screened for relevance. Documents that were not obviously ineligible or irrelevant (based on the abstract review) were retrieved from the Vanderbilt University Libraries, Interlibrary Loan, ERIC, University Microfilms, and government documents sources for final eligibility screening. Final determination of eligibility for all studies irrespective of source was made from the full study report document(s) on the basis of detailed eligibility criteria. The abstract screening and eligibility determination were performed by the first author or a trained research assistant. Any ambiguities or questions about eligibility were resolved through discussion by both authors.

Although some research suggests that two reviewers might increase accuracy in identifying potentially eligible studies from abstracts obtained through bibliographic searches (Edwards, et al., 2002), our experience is that most abstracts do not provide enough detail to allow reviewers to make reliable judgments about whether a study meets the review criteria. Thus, we reviewed abstracts mainly to eliminate those clearly irrelevant and deferred the final determination until the entire study report was screened. While this required retrieving many more documents than eventually ended up in the review, it allowed us to make eligibility decisions based on the most complete information about a study that was available.

Data Management and Extraction

The coding protocol used for this project allowed for coding of a wide variety of study characteristics, as well as study results. The standardized mean difference effect size statistic (Cohen, 1988; Lipsey & Wilson, 2001) was used to record intervention effects. This effect size statistic is defined as the difference between the treatment and control group means on an outcome variable divided by their pooled standard deviations. When means and standard deviations were not available, effect sizes were estimated from the statistics that were reported using the procedures described in Lipsey and Wilson (2001). Typically, studies reported results on multiple outcome constructs (e.g., aggression, social skills, school achievement) and often included more than one operationalization of the same construct (e.g., parent and teacher reports of aggressive behavior). For purposes of this systematic review, all effect sizes that assessed aggressive or disruptive behavior outcomes that could be extracted from a study were coded. Procedures for maintaining statistical independence during analysis in cases of multiple effect sizes will be discussed in the next section.

In addition to effect size values, information was coded for each study that describes the methods and procedures, the intervention, and the subject samples. The items describing study methods and procedure include details of the design, measures, and attrition. Those coded to describe the subject samples include age, gender, ethnicity, socioeconomic status, and risk for later antisocial behavior. The interventions were described by coding the details of specific program components; duration, intensity, setting, and format of the program; delivery personnel; and, other such characteristics.

All study coding was done on computer screens configured in FileMaker Pro^® for direct entry into the database. All coding was completed either by the first author or by trained research assistants. Any study coded by a research assistant was reviewed for accuracy by the first author. Any discrepancies or ambiguities in coding were resolved by the two authors.

Statistical Procedures

Effect sizes based on small samples are known to be biased; to adjust for this, all effect sizes were multiplied by the small sample correction factor, 1 – (3/4n-9), where n is the sample size for the study (Hedges & Olkin, 1985). Also, each effect size was weighted by its inverse variance in all computations so that its contribution was proportionate to its reliability (Hedges & Olkin, 1985). Examination of the effect size distribution identified a small number of outliers with potential to distort the analysis; these were recoded (i.e., Winsorized) to less extreme values (Hedges & Olkin, 1985; Lipsey & Wilson, 2001). In addition, several studies had unusually large samples. Because the inverse variance weights chiefly reflect sample size, those few studies would dominate any weighted analysis in which they were included. That in itself would not be problematic if we could be assured that sample size was not related to other study characteristics, e.g., type of intervention, quality of implementation, etc., that would, therefore, also be heavily weighted in any analysis. In our experience, however, sample size is confounded with other study characteristics in ways that can distort the analysis. We therefore recoded the extreme tail of the sample size distribution to a maximum of 250 subjects per intervention or control group for the computation of weights.3 These adjustments to outliers allow us to retain them in the analysis with relatively extreme values, but make those values less extreme so that they do not exercise highly disproportionate influence on the analysis results.

The intervention outcome of primary interest was aggressive and disruptive behavior, which involves a variety of negative interpersonal behaviors including fighting, hitting, bullying, verbal conflict, disruptiveness, acting out and the like.4 The most common type of measure was a teacher-report questionnaire, though many studies provided results for more than one aggressive behavior outcome. To create sets of independent effect size estimates for analysis, only one effect size from each subject sample was used in any analysis. The procedures for selecting an independent set of effect sizes are described below.

Finally, many studies provided data sufficient for calculating mean difference effect sizes on the outcome variables measured at the pretest. In cases where pretest effect sizes were available, we adjusted the posttest effect sizes for pretest differences by subtracting the pretest value from the posttest value. In the analyses presented below, we tested whether there were systematic differences between effect sizes that were adjusted and those that were not by including dummy codes for adjustment in the analyses.

Description of Eligible Studies

The search strategy identified 89 eligible reports, which gave the results of 73 unique research studies of universal school-based social information processing programs. Some eligible studies were reported in multiple reports, while some reports presented results for multiple independent studies. In addition, when the results of a study were reported for subgroups (e.g., male and female results were reported separately), we coded the two subgroups rather than the aggregate and treated them as two studies. Table 1 summarizes the characteristics of the 73 eligible studies of universal social information processing programs. Coding details for each of the 73 studies in the review can also be found in the Appendix.

The majority of programs were conducted in the United States since 1990. Two-thirds were published in peer reviewed journals. The subjects in the universal programs represented a cross-section of school children. Most samples were an even mix of boys and girls, as would be expected with universal programs that are delivered to entire classrooms; age ranged from 4 to 16. A range of racial and ethnic groups was represented. Nearly half of the samples were comprised of predominantly low socioeconomic status children. Fifteen percent of the programs were delivered in special education settings; children in the special education classrooms and alternative schools all had behavior problems or similar social difficulties (in addition to some having academic difficulties), but the programs were delivered universally to all children in those classrooms. Among the studies conducted in regular schools, two elected to present results only on the participating children with behavior problems. They are included here (rather than in the companion review of selected/indicated programs) because the intervention was delivered universally and the behavior problem children were not separately identified for treatment, only for outcome measurement.

Only six studies used individual random assignment to place subjects in treatment and control groups. This is not unexpected since the social information processing programs reviewed here were all delivered to entire classrooms. Most studies assigned whole groups (classes or schools) to receive the program or serve as comparison groups. Of these, most were assigned using non-random methods. Eighty-six percent of studies presented pretest results for the outcome variable, which allowed us to adjust for any pretest differences between groups at the effect size level.5 Overall, the average pretest effect size was significantly negative, indicating that control groups were generally more aggressive than intervention groups before treatment began. However, the pretest effect sizes for non-randomized studies were not significantly different from those for randomized studies.

Seven programs were coded as routine practice programs (administered by school personnel as an ongoing school program). The remaining programs were conducted by researchers primarily for research or demonstration purposes. Correspondingly, the program evaluators/researchers tended to be involved in treatment delivery or supervision of delivery personnel. In 45% of the studies, the primary researcher developed the program under study, while in a third of the studies the program developer was not involved in the research project. For just over 22% of the studies, program developers were not involved, but the primary researchers modified the program. We believe this may engender some ownership in the program; in a sense, the researcher becomes a program developer as a result of modifying an existing program.

As would be expected for universal programs, teachers were the most common delivery personnel. Programs were relatively short term, with nearly two-thirds being less than 17 weeks. The frequency of sessions ranged from less than weekly for some of the longer programs, to daily. While the reports for most programs identified no implementation problems, over a third mentioned some difficulty implementing the intervention as intended. Those problems tended to involve not delivering all components of the program, or having teachers or schools drop out of the study before posttest data collection. Finally, the most common foci of the programs were social problem solving, perspective taking, and anger management. Behavioral social skills training was included in 32% of the programs.

Mean Effects of Universal Social Information Processing Programs

The 73 eligible studies generated a total of 162 standardized mean difference effect sizes on aggressive or disruptive behavior; these effect sizes index the posttreatment differences between the treatment and comparison groups. Most studies (51) generated more than one effect size for aggressive behavior (ranging from two to ten), while 22 studies provided only a single effect size for aggressive behavior. The multiple measures of aggressive and disruptive behavior within studies were either different aspects of aggressive behavior (e.g., physical and non-physical aggression) or were the same type of aggressive behavior reported by different informants.

To create a set of independent effect sizes for analysis, a combination of procedures was used. When studies reported results on different types of aggressive behavior, we selected the effect size(s) that most closely represented interpersonal physical aggression and discarded the others. Next, we wanted to retain informant as a variable for analysis, so did not elect to average across effect sizes from different informants when more than one was reported. If there was more than one effect size from the same informant or source in the same study, however, their mean value was used. Then, from the studies with multiple effect sizes reported by different informants, one effect size was selected from the informant that was most frequently represented in the data. In this case, teachers were the most common informants, followed by self-reports from the participants themselves. These procedures resulted in the 73 effect sizes that are the subject of the following analyses.

The overall random effects mean was .21 (p<.001), indicating that subjects in the treatment groups had significantly lower aggressive and disruptive behavior than comparison subjects after participating in universal social information processing programs. Figure 1 shows the forest plot for the effect size distribution, using random effects methods. The effect sizes range from -.58 to 1.48. This figure shows that the majority of effects (over 75%) are positive, though not large. Note that the inverse variance weights used to compute the weighted mean effect size and the confidence intervals around the effect size values are computed using the subject-level sample size values on which each effect size is based. Because many of the studies used groups (e.g., classrooms, schools) as the unit of assignment to intervention and control conditions, they involved a design effect associated with the clustering of students within classrooms or schools that reduces the effective sample size. We have no basis for estimating those design effects or adjusting the inverse variance weights for them, so they were ignored in the analyses reported here. This should not greatly affect the effect sizes estimates or the magnitude of their relationships to moderator variables, but will inflate the statistical significance levels and underestimate the confidence intervals.

Analysis of Moderator Effects

There was also significant variability across the different studies in the effect sizes. A test of the homogeneity of the effect sizes using the Q-statistic (Hedges & Olkin, 1985) showed more variability across studies than expected from subject-level sampling error (Q72=199, p<.001). This variation was expected to be associated with the nature of the interventions, subjects, and methods in the studies of universal programs. Our next step, therefore, was to identify the study characteristics most strongly associated with effect size.

OPEN IN VIEWER

1

Post-treatment Effect Sizes and Confidence Intervals for Aggressive & Disruptive Behavior Outcomes