The Effects of School-Based Social Information Processing Interventions on Aggressive Behavior,Part II: Selected/Indicated Pull-Out Programs

Social Information Processing

The programs included in this systematic review all attempt to improve one or more aspects of students’ social information processing. Under this intervention model, social behavior is the result of six interrelated steps: (1) encoding situational and internal cues, (2) interpretation of cues, (3) selecting or clarifying a goal, (4) generating or accessing possible responses, (5) choosing a response, (6) and behavioral enactment (Dodge; 1986; Crick & Dodge, 1994). Negative social behavior such as aggression is thought to be the result of cognitive deficits at one or more of these stages.1 For some children, the inability to process social information results in inappropriate behavioral responses and aggressive children tend to differ from non-aggressive children in various stages of social information processing (Dodge, Pettit, McClaskey, & Brown, 1986; Kendall, 1995). To illustrate, deficits at the encoding or interpretation stage of processing may involve misinterpreting as hostile the intent of others in neutral or ambiguous social situations. Hostile misattributions have been linked to aggressive responses (Crick & Dodge, 1994). And, aggressive children are more likely to make hostile attributions than their non-aggressive peers (Slaby & Guerra, 1988). Deficits at the goal selection or clarification stage can result in the selection of antisocial rather than prosocial goals (Asher & Renshaw, 1981; Crick & Dodge, 1989). Children who have difficulty accessing or evaluating responses to social situations (Steps 4 and 5) tend to have fewer responses from which to choose in social situations and may fail to evaluate the consequences of particular behaviors (e.g., Mize & Cox, 1990; Spivack & Shure, 1974).

Social Information Processing Programs

Some of the earliest programs designed to address social information processing deficits were the social problem solving programs developed in the 1970s (e.g., D’Zurilla & Goldfried, 1971; Shure & Spivack, 1972; 1979). These programs were designed to improve social behavior by teaching cognitively-based problem solving skills. For example, the I Can Problem Solve program (Shure, 1992) involves teaching participants a series of problem solving skills that includes perspective taking, alternative response generation, consequential or means-ends thinking, and causal thinking.

In more recent years, many variations on this theme have been developed (Coleman, Wheeler, & Webber, 1993). Attribution retraining programs, such as Hudley's BrainPower (Hudley, 1991) program, focus on the early stages of the social information processing model and teach children to accurately detect intentionality, to make nonhostile attributions when social encounters are ambiguous, and to generate appropriate behavioral responses to ambiguous negative situations. Lochman's Anger Coping program (Lochman, Lampron, Burch, & Curry, 1985) includes a module that helps children learn cognitive goal-setting skills, a social problem solving component, and a module on anger control (which focuses on the emotional aspects of social information processing).

While these are examples of some typical social information processing programs, a conceptual definition of the programs in general is necessary for conducting the proposed systematic review. This definition aids in identifying candidate programs for the review and in coding the particular treatment components of each program. For purposes of this review, therefore, a social information processing program is one with the following distinct characteristics:

The program involves training in one or more of the social information processing steps: (1) encoding situational and internal cues, (2) interpretation of cues, (3) selecting or clarifying a goal, (4) generating or accessing possible responses, (5) choosing a response, (6) and behavioral enactment.

What Social Information Processing Programs are Not

Another set of popular programming strategies also focuses on social competence. These programs, often called by the generic term behavioral social skills, are distinct from the social information processing programs that are the focus of this review. Behavioral social skills curricula focus primarily on the social (or antisocial) behaviors themselves, rather than the underlying cognitive thought processes. These programs generally teach specific skills such as making eye contact, smiling in context, paying compliments, communication skills, group entry skills, assertiveness, and the like (see Goldstein & Pentz, 1984 for a review).

In addition, there are other cognitively-oriented programs that do not specifically target social information processing. For example, programs for children with attention or activity level difficulties often involve teaching skills for cognitive impulse control. Similarly, there are cognitively oriented programs for dealing with stress, divorce, depression, etc. (e.g., Alpert-Gillis, Pedro-Carroll, & Cowen, 1987; Lewinsohn, Clarke, Hops, & Andrews, 1990). While these other cognitive programs focus on thinking skills, they do not have the focus on interpersonal relationships that is characteristic of social information processing programs.

Selected and Indicated Prevention

Programs that address social information processing difficulties tend to be structured and have detailed lesson plans, which make them attractive to schools. These programs are also easily delivered by teachers or school psychologists and can be used in different formats (group or individual) and settings (classrooms or out-of-class school facilities). Our original intent for this systematic review was to include all research studies of social information processing programs delivered in school settings. As we began collecting the relevant research literature, however, we discovered that the programs were delivered using several different approaches to prevention. In particular, social information processing programs are sometimes delivered universally to all students and, other times, only to specially selected at-risk or behavior problem students. That is, social information processing programs can be universal, selected, or indicated (Mrazek & Haggerty, 1994). Universal prevention interventions are delivered in classroom settings to an entire classroom without regard to the risk level or degree of behavior problems among students in the classroom. Selected and indicated prevention interventions target children who are exhibiting risk factors (such as anger or hyperactivity) or children who are already showing early signs of disorder (i.e., behavior problems). The studies of social information processing programs that focus on universal prevention, of course, tend to have different student subjects than studies of programs that focus on youth selected or indicated on the basis of various risk factors. In addition to the differences in the subject samples in universal versus selected or indicated programs, we noticed that there were other important differences between the two prevention approaches, most notably in study procedure and method. Nearly all of the studies of programs delivered under the selected/indicated approach used random assignment of individual subjects to produce treatment and comparison groups, while only a few studies of universal programs used individual randomization. Based on these differences in study and subject characteristics, and the possibility that these factors might be associated with study outcomes, we elected to conduct two separate systematic reviews. The review presented here focuses solely on selected and indicated social information processing programs delivered in school settings. A second review focuses on school-based universal social information processing programs (Wilson & Lipsey, forthcoming).

Prior Research on Social Skills Interventions

Several recent narrative reviews of social skills interventions are available (Clayton, Ballif-Spanvill, & Hunsaker, 2001; Howard, Flora, & Griffin, 1999; Moote, Smyth, & Wodarski, 1999; Nangle, Erdley, Carpenter, & Newman, 2002) but these cover social skills interventions quite broadly by including training in social skills, friendship making skills, communication skills, and other such social behaviors without an explicit focus on altering cognitive and social information processing deficits. A few narrative reviews of programs specific to social information processing have also been conducted (e.g., Coleman, Wheeler, & Webber, 1993; Pellegrini & Urbain, 1985), but these are somewhat dated and also include programs delivered to clinical populations in non-school settings. Several meta-analysis of related interventions are available (e.g., Beelmann, Pfingsten, & Lösel, 1994; Denham & Almeida, 1987; Lösel & Beelmann, 2003; Mytton, DiGuiseppi, Gough, Taylor, & Logan, 2002; Quinn, Kavale, Mathur, Rutherford, & Forness, 1999; Schneider & Byrne, 1985). Like the narrative reviews, but with the exception of Denham and Almeida (1987), these syntheses focus on social competence programs in general and thus include both social information processing and behavioral social skills programs. The Denham and Almeida meta-analysis focuses exclusively on interpersonal cognitive problem solving interventions, but is 15 years old and does not include international studies.

Criteria for Including Studies in the Review

Interventions. The interventions were required to meet the three criteria that define social information processing programs. Although other treatment components were allowed to be present (e.g., behavioral social skills training, parenting skills training), the social information processing component needed to be the clear focus of the program. The definitional criteria are as follows:

Training was provided on one or more of the social information processing steps: (1) encoding situational and internal cues, (2) interpretation of cues, (3) selecting or clarifying a goal, (4) generating or accessing possible responses, (5) choosing a response, (6) and behavioral enactment.

Settings and Subjects. The interventions had to be delivered to school-aged children (K-12 or equivalent ages in international settings) in regular school settings during school hours. Moreover, the intervention had to be delivered to children who were specially selected for treatment by virtue of any risk factor linked to later antisocial behavior or for evident behavior problems. Special education classrooms and alternative schools were eligible school settings, although classrooms in residential facilities (e.g., psychiatric hospitals) were not. After-school programs were not eligible. Any qualifying school in any region or country was eligible.

Outcomes. The study reported intervention effects for at least one outcome variable, measured on children, representing aggressive behavior, broadly defined to include violence, aggression, fighting, person crimes, disruptive behavior problems, acting out, conduct disorder, externalizing problems, and so forth.

Study Design. Only studies using a control group design were eligible. The intervention and control groups could be randomly or nonrandomly assigned but, if nonrandom, needed to be matched or provide evidence of initial equivalence on key demographic variables and/or pretests. Control groups could represent placebo, wait-list, no treatment, or “treatment as usual” conditions. Studies without control or comparison groups were not eligible. This included one-group pretest-posttest studies and studies in which a treatment condition was compared to another treatment condition.

Search Strategy for Identification of Studies

An attempt was made to identify and retrieve the entire population of empirical studies that met the eligibility criteria specified above, including both published and unpublished studies. Several sources were used to identify potentially eligible research reports. First, a large database compiled at the Center for Evaluation Research and Methodology with NIMH grant funding (Lipsey, PI) was searched for eligible interventions. That database includes studies of early intervention programs targeting a wide range of risk factors for antisocial behavior and includes many school-based programs.

In addition, a comprehensive search of bibliographic databases, including Psychological Abstracts, Dissertation Abstracts International, ERIC (Educational Resources Information Center), the Campbell and Cochrane Collaboration trials registers, U.S. Government Printing Office publications, National Criminal Justice Reference Service, and MedLine was conducted, with a special focus on recent studies not included in the existing database.

The search strategy utilized three categories of search terms2 derived from the controlled vocabulary used to index articles for each specific database. Wildcard characters were used to identify variants of words (e.g., delinquen* to locate delinquent, delinquents, and delinquency). Within each category described below, search terms are connected by the OR operator; the results for each of the three categories were combined with the AND operator:

The population of interest: child, adolescent, childhood (MEDLINE); children, adolescents, preadolescents, students (ERIC); children, adolescents, students (PsychInfo), and children, adolescents, students (Sociological Abstracts);

Third, the bibliographies of previous meta-analyses and literature reviews (e.g., Denham & Almeida, 1987; Durlak, 1997) were inspected for studies that met the eligibility criteria. In addition, the bibliographies of retrieved studies were themselves examined for potentially eligible research reports. Finally, follow-up searches on the first and second authors of all eligible studies and cited reference searches of eligible articles in the Social Sciences Citation Index were conducted. We have found this forward searching technique to be more efficient and fruitful than conducting hand searches of journal issues.

Studies identified through our searches were retrieved from the library, obtained via interlibrary loan, or requested directly from the author(s). In addition, we attempted to locate and include any eligible studies published in non-English language sources that were missed by the search strategies described above by asking colleagues from both inside and outside the U.S. to assist us with locating foreign-language studies.

Selection of Studies

Abstracts of the studies found through the search procedures were screened for relevance. Documents that were not obviously ineligible or irrelevant (based on the abstract review) were retrieved from the Vanderbilt University Libraries, Interlibrary Loan, ERIC, University Microfilms, and government documents sources for final eligibility screening. Final determination of eligibility for all studies irrespective of source was made from the full study report document(s) on the basis of detailed eligibility criteria. The abstract screening and eligibility determination were performed by the first author or a trained research assistant. Any ambiguities or questions about eligibility were resolved through discussion by both authors.

Although some research suggests that two reviewers might increase accuracy in identifying potentially eligible studies from abstracts obtained through bibliographic searches (Edwards, et al., 2002), our experience is that most abstracts do not provide enough detail to allow reviewers to make reliable judgments about whether a study meets the review criteria. Thus, we reviewed abstracts mainly to eliminate those clearly irrelevant and deferred the final determination until the entire study report was screened. While this required retrieving many more documents than eventually ended up in the review, it allowed us to make eligibility decisions based on the most complete information about a study that was available.

Data Management and Extraction

The coding protocol used for this project allowed for coding a wide variety of study characteristics, as well as study results. The standardized mean difference effect size statistic (Cohen, 1988; Lipsey & Wilson, 2001) was used to record intervention effects. This effect size statistic is defined as the difference between the treatment and control group means on an outcome variable divided by their pooled standard deviations. When means and standard deviations were not available, effect sizes were estimated from the statistics that were reported using the procedures described in Lipsey and Wilson (2001). Typically, studies reported results on multiple outcome constructs (e.g., aggression, social skills, school achievement) and often included more than one operationalization of the same construct (e.g., parent and teacher reports of aggressive behavior). For purposes of this systematic review, all effect sizes that assessed aggressive or disruptive behavior outcomes that could be extracted from a study were coded. Procedures for maintaining statistical independence during analysis in cases of multiple effect sizes will be discussed in the next section.

In addition to effect size values, information was coded for each study that describes the methods and procedures, the intervention, and the subject samples. The items describing study methods and procedures include details of the design, measures, and attrition. Those coded to describe the subject samples include age, gender, ethnicity, socioeconomic status, and risk for later antisocial behavior. The interventions were described by coding the details of specific program components; duration, intensity, setting, and format of the program; delivery personnel; and, other such characteristics.

All study coding was done on computer screens configured in FileMaker Pro^® for direct entry into the database. All coding was completed either by the first author or by trained research assistants. Any study coded by a research assistant was reviewed for accuracy by the first author. Any discrepancies or ambiguities in coding were resolved by the two authors.

Statistical Procedures

Effect sizes based on small samples are known to be biased; to adjust for this, all effect sizes were multiplied by the small sample correction factor, 1 – (3/4n-9), where n is the sample size for the study (Hedges & Olkin, 1985). Also, each effect size was weighted by its inverse variance in all computations so that its contribution was proportionate to its reliability (Hedges & Olkin, 1985). Examination of the effect size distribution identified a small number of outlier effect sizes with potential to distort the analysis; these were recoded (i.e., Winsorized) to less extreme values (Hedges & Olkin, 1985; Lipsey & Wilson, 2001). These adjustments retain the effect sizes in the analysis with relatively extreme values, but make those values less extreme so that they do not exercise highly disproportionate influence on the analysis results.3

The intervention outcome of principal interest was aggressive and disruptive behavior, which involves a variety of negative interpersonal behaviors including fighting, hitting, bullying, verbal conflict, disruptiveness, acting out and the like.4 The most common type of measure was a teacher-report questionnaire, though many studies provided results for more than one aggressive behavior outcome. To create sets of independent effect size estimates for analysis, only one effect size from each subject sample was used in any analysis. The procedures for selecting an independent set of effect sizes are described below.

Finally, many studies provided data sufficient for calculating mean difference effect sizes on the outcome variables measured at the pretest. In cases where pretest effect sizes were available, we adjusted the posttest effect sizes for pretest differences by subtracting the pretest value from the posttest value. In the analyses presented below, we tested whether there were systematic differences between effect sizes that were adjusted and those that were not by including dummy codes for adjustment in the analyses.

Description of Eligible Studies

The search strategy identified 68 eligible reports, which reported the results of 47 unique research studies. Some eligible studies were reported in multiple reports, while some reports presented results for multiple independent studies. In addition, when the results of a study were reported for subgroups (e.g., male and female results were reported separately), we coded the two subgroups rather than the aggregate and treated them as two studies. Table 1 summarizes the characteristics of the 47 selected and indicated social information processing programs included in this review. Coding details for each of the 47 studies can be found in the Appendix.

Nearly 90 percent of the studies were conducted in the United States, and slightly over half were published in peer reviewed journals. Studies were published from the 1970s to the present, with most programs in the 1980s and 1990s. As would be expected with populations at risk for aggressive behavior, the samples were predominantly male. Age ranged from six to 16, with 45% of the samples in the nine to eleven year range. In the United States, nine to eleven year olds are typically in upper elementary school, between the 4^th and 6^th grades. Among the studies that reported the ethnicity for their subject samples, over half were comprised primarily of minority youth. Forty percent of the studies were conducted with predominantly low income populations. In terms of the risk status of student participants, 32% of the studies focused on selected groups and 64% focused on indicated groups. There were two studies with very low risk children. They are included here because the children were specially selected and pulled out of their regular classrooms for treatment.

The majority (74%) of studies used random assignment of individual students to create treatment and comparison groups. Eighty-seven percent of the studies presented pretest results on the outcome variable, which allowed us to adjust for any pretest differences between groups at the effect size level.5 Overall, the average pretest effect size was not significantly different from zero, indicating the treatment and comparison groups had equivalent levels of aggressive behavior before treatment began. In addition, the random and nonrandom assignment studies were not significantly different at the pretest.

Only one program was coded as a routine practice program (administered by school personnel as an ongoing school program). The remaining programs were conducted primarily for research or demonstration purposes. Correspondingly, the program evaluators/researchers tended to be involved in treatment delivery or supervision of delivery personnel. In 55% of the studies, the primary researcher developed the program under study, while there were only 5 studies in which the program developer was not involved in the research project. For just over a third of the studies, program developers were not involved, but the primary researchers modified the program. We believe this may engender some ownership in the program; in a sense, the researcher becomes a program developer as a result of modifying an existing program. In addition, because most of the programs had heavy researcher involvement in treatment delivery and program administration, there were few studies (9) in which implementation problems were reported.

Programs were mainly conducted using group formats in regular schools, though seven programs were either solely one-on-one or included individual as well as group sessions. Five programs were conducted with special education students. Treatment duration ranged from short programs of less than 8 weeks to those lasting one school year or more. The majority of programs had sessions once or twice a week. Since students were typically pulled out of their regular classrooms to meet for treatment, it is not surprising that most programs could only arrange sessions once or twice a week. A range of delivery personnel was utilized, with most having some affiliation with the researchers. Finally, Table 1 shows the frequency of the different types of social information processing interventions utilized. Most programs involved more than one aspect of social information processing, and 13 also included a behavior modification component.

Mean Effects of Selected and Indicated Social Information Processing Programs

The 47 eligible studies generated a total of 120 standardized mean difference effect sizes on aggressive or disruptive behavior; these effect sizes index the posttreatment differences between the treatment and comparison groups. Most studies (32) generated more than one effect size (ranging from two to nine), while 15 studies provided only a single effect size. The multiple measures of aggressive and disruptive behavior within studies were either different aspects of aggressive behavior (e.g., physical and non-physical aggression) or were the same type of aggressive behavior reported by different informants.

To create a set of independent effect sizes for analysis, a combination of procedures was used. When studies reported results on different types of aggressive behavior, we selected the effect size(s) that most closely represented interpersonal physical aggression and discarded the others. Next, we wanted to retain informant as a variable for analysis, so did not elect to average across effect sizes from different informants when more than one was reported. If there was more than one effect size from the same informant or source, however, their mean value was used. Then, from the studies with multiple effect sizes reported by different informants, one effect size was selected from the informant that was most frequently represented in the data. In this case, teachers were the most common informants, followed by self-reports from the participants themselves. These procedures resulted in the 47 effect sizes that are the subject of the following analyses.

The overall random effects mean was .26 (p<.001), indicating that subjects in the treatment groups had significantly lower aggressive and disruptive behavior than comparison subjects after participating in selected or indicated social information processing programs. Figure 1 shows the forest plot for the effect size distribution, using random effects methods. The effect sizes range from -.71 to 1.29. This figure shows that the majority of effects (over 60%) are positive, though not large.

Standardized Mean Difference Effect Sizes for Aggressive and Disruptive Behavior

OPEN IN VIEWER

1

Post-trentine ut Effect Sizes and Confidence Intervals for Aggressive it Disruptive Behavior Outcomes

Analysis of Moderator Effects

There was also significant variability across the different studies in the effect sizes. A test of the homogeneity of the effect sizes using the Q-statistic (Hedges & Olkin, 1985) showed more variability across studies than expected from subject-level sampling error (Q46=97, p<.001). This variation was expected to be associated with the nature of the interventions, subjects, and methods in the studies of universal programs. Our next step, therefore, was to identify the study characteristics most strongly associated with effect size.