PROTOCOL: On the Malleability of Self-Control: Theoretical and Policy Implications Regarding a General Theory of Crime

The Current Study

There has been much attention paid in both criminology and psychology with respect to the importance of self-control in regulating antisocial, delinquent, and criminal behavior over the life course. Given the importance of self-control, there have also been several programmatic efforts designed to improve self-control among children and adolescents. In an effort to build the knowledge base in this area, this study asks two critical questions: (1) What is the effectiveness of programs designed to improve self-control up to age 10 among children and adolescents?; and (2) What are the effects of these programs on self-control and delinquency/crime? Examining both self-control and delinquency outcomes would provide a comprehensive review that identifies a large number of studies and will likely evince a sounder conclusion and inform policy proscription for the general theory of crime. This meta-analysis, then, focuses on two inter-related outcomes: (1) What are the effects of self-control improvement programs up to age 10 for improving self-control among children/adolescents (self-control as the dependent variable)?; and (2) What are the effects of self-control improvement programs on delinquency outcomes (delinquency as the dependent variable).

Criteria for Inclusion and Exclusion of Studies in the Review

Studies that investigated the effects of self-control improvement programs on child behavior problems such as conduct problems, antisocial behavior and delinquency were included. Following the more general systematic (e.g., Campbell) reviews, studies were only included if they had a randomized controlled trial design with post-test measures of self-control and/or child behavior problems for the experimental and control participants. ¹ The preliminary eligibility criteria are as follows: 1. Types of Studies: The study must have used a randomized controlled experimental design; 2. Types of Participants: The review was primarily focused on children ages 10 and under or the mean age of the sample was no greater than age 10 at the start of the intervention. Studies with mentally and/or physically handicapped subjects were not included; 3. Type of Intervention: Studies were eligible for this review when self-control improvement was a major component of the intervention; 4. Types of Outcomes: The study must have included at least one child-based outcome measure of self-control ² and/or at least one child-based behavioral outcome measure of general behavior problems including antisocial behavior and delinquency ³ ; 5. Sufficient Data: The study had to provide adequate post-test data for calculating an effect size if one was not provided (i.e., means and standard deviations, t-tests, F-tests, p-values, etc.); 6. There is no restriction to time frame; 7. There are no geographic restrictions; 8. Both published and unpublished reports were considered; 9. Qualitative studies were not included; and 10. Studies needed to be published in English.

Search Strategy for Identification of Relevant Studies

Several strategies were used to perform an exhaustive search for literature fitting the eligibility criteria: (1) A keyword ⁴ search was conducted across a number of online abstract databases ⁵ ; (2) The reference lists of previous reviews of early childhood prevention/intervention programs in general and self-control improvement programs specifically were consulted (Aos et al., 2004, 2006; Karoly et al., 1998; Greenwood et al., 2006; Suhodolsky et al., 2004; Farrington & Welsh, 2007); (3) Hand searches were carried out on leading journals in the field ⁶ ; (4) The publications of research and professional agencies were searched ⁷ ; and (5) Recognized scholars (experts) in various disciplines who were knowledgeable in the specific area of self-control improvement programs were contacted. These scholars were defined as those who authored two or more studies that appear on our inclusion list.

Several strategies were used to obtain full-text versions of the studies found through the searches of the various abstract databases. First, we attempted to obtain full-text versions from the electronic journals available through several university library systems. When electronic versions were not available, we used print versions of journals available at the library. If the journals were not available at the university libraries, we used the Interlibrary Loan System (ILL) to try to obtain the printed version from the libraries of other institutions. In the case where these methods failed, we then made attempts to contact the author(s) of the article and/or the agency that funded the research to try to obtain a copy of the full-text version of the study.

Details of Study Coding Categories

All eligible studies were coded (see protocol in Appendix A) on a variety of criteria such as reference information (title, authors, publication year, etc.); nature of description of selection of sample, outcomes, etc.; nature and description of control group; unit of analysis; sample size; a description of the self-control improvement intervention; reports of statistical significance (if any); and effect sizes (if any). One investigator independently coded each eligible study, and consulted with co-authors regarding the final coding decisions. Further, we attempted to assess the quality of the studies in terms of research design, sample bias, and attrition bias.

Criteria for Determination of Independent Findings

It is the case that most outcome studies rely on multiple measures, but there is disagreement as to how this issue should be handled with some researchers opting to use only one outcome source over another for reasons such as teacher ratings are likely to be less biased than parent reports and systematic “unbiased” observer ratings may be more accurate than teacher ratings (Farrington & Welsh, 2003). Other meta-analyses have averaged the effect sizes (ESs) across outcome measures and outcome sources when generating an individual effect size for each study (McCart et al., 2006). Still, others have noted that this method may lead to the loss of important information and create some difficulty when interpreting the overall effect (Casey & Berman, 1985).

In light of the apparent controversy over which method is more appropriate, we adopted a method of compromise, and report a series of effect sizes by outcome measure (e.g., self-control and delinquency) and outcome source (e.g., parent report, teacher report, direct observation, self-report, and/or clinical report). Further, if a study included more than one treatment condition, then only the treatment condition that used a self-control improvement program was used to generate the relevant ESs. In addition, in the case where multiple control groups exist, then only the outcomes for the no-treatment control group (or wait-list control group) were used to calculate the ES. Similarly, when multiple treatment groups existed where each treated group received a self-control improvement program, then only one ES was calculated for the study by averaging the mean and standard deviation across the treatment groups and then comparing this one pooled mean and standard deviation to that of the control group in order to generate the ES for the study. As one more method for ensuring the statistical independence of findings, we calculated only one single ES for one particular sample in the event that multiple studies reported findings from the same sample of treated youth.

Analytic Procedures

We rely on Cohen's (1988) d for determining the effect sizes for this meta-analysis. The main source of information for calculating Cohen's d was the standardized mean difference, but in situations where means and standard deviations were not provided t-values, f-values, p-values, partial r etc. was used to calculate the effect sizes (see Lipsey & Wilson 2001 for the relevant formulas). Hedges and Olkin (1985) recommend calculating an unbiased ES that accounts for the discrepancy between the sample ES and the population ES. These authors also suggest that an ES of a small sample study does not have as much “impact” on the overall ES as does an ES calculated from a large sample study. As such, they recommend using inverse variance weights when performing a meta-analysis. Therefore, we used the Hedges and Olkin adjustment and inverse variance weights when determining the ESs in the analysis.

All of the meta-analysis results were estimated using Lipsey and Wilson's SPSS macros relying on a random effects model using inverse variance weight methods. It is also our general assumption that the individual ESs were not likely to be homogeneous so we estimated a series of moderator analyses using Lipsey and Wilson's SPSS analog to the ANOVA macro. Some of the relevant variables that are included in the moderator analyses include publication year, country of publication, small/large samples, published/not published, treatment type, treatment modality (group/individual), treatment duration, and treatment setting. The last stage of the analysis presents the results from a weighted least squares regression model (estimated with inverse variance weights and random effects) where the variables mentioned above are included as predictors of the ES. Publication bias is also evaluated using traditional methods including a comparison of the mean effect size for published/unpublished studies and an investigation of publication bias with a funnel plot and associated test statistics (e.g., Kendall's test and Egger's test) estimated with the ‘metafunnel’ macro available in Stata.

Types of Interventions

Considering the variability of the self-control improvement interventions, it is important to discuss some examples of the broad categories of intervention type. The most recognizable of the social skills development programs are studies of the Conduct Problems Prevention Research Group (CPPRGa, 1999) and Tremblay et al.'s (1991) Montreal Youth Study. The social skills development intervention in the CPPRG study is called Fast Track and uses a “unified model of prevention” where a number of integrated intervention programs are applied such as: curriculum, parent groups, child social skills training groups, parent-child sharing time, home visiting, child peer pairing, and academic tutoring. The program involves lessons addressing four domains of skills: (1) skills for emotional understanding and communication; (2) friendship skills; (3) self-control skills; and (4) social problem solving skills (CPPRGa, 1999, p. 635). Comparatively, Tremblay et al.'s intervention also involved multiple program components, but one of these core competencies involved social skills training and was administered within small groups of prosocial peers. Another key component of Tremblay et al.'s intervention was self-control improvement sessions developed around themes such as “look and listen,” “following rules,” “what to do when I am angry,” “what to do when they do not want to play with me,” and “how to react to teasing” (p. 154).

Jackson and Calhoun's (1982) study was classified as a cognitive coping strategies intervention, which involved “cognitive self-instructional training where children are taught to covertly emit verbalizations that will cue or guide their non-verbal behavior” (Jackson & Calhoun, 1982, p. 7). Similarly, Reid and Borkowski (1987) versions of cognitive coping strategies focuses on using psychoeducational tasks where an instructor verbalizes correct self-control statements such as “find out what I am supposed to do,” “consider all answers,” “stop and think,” “mark my answer,” and “check my answer” while performing various tasks, and then has the child repeat these steps and verbalize these statements while performing similar tasks.

Toner et al. (1978) is an example of a study classified as a video tape training/role playing intervention. Here, the children are sat in front of a television and told by the instructor: “Here is my television. The boy you will see on TV has been told not to touch the toys that are in front of him. Watch closely” (p. 285). During the course of watching the video, the boy in the video would either do things appropriately or be resistant to commands at times. At each response time (whether appropriate or resistant), the subject was asked whether the boy's response in the video was correct. If the subject replied with an affirmative response, then the video continued. Following the video tape training, the subject was also left alone for a period of time and their behavior and self-control was observed. Baggerly (1999) is another example of a video tape training/role playing intervention where didactic lectures, experiential activities (e.g., role playing), and viewing videos of child-centered play sessions were used with the intention of improving the children/adolescents’ self-control. The children /adolescents in this particular study received the training for 35 minutes twice a week for five weeks and then once a week for the remaining five weeks.

The immediate/delayed rewards clinical interventions can best be characterized by Mischel and Baker (1975). This type of intervention took place in an experimental room where the room was divided by a wooden barrier where there were battery operated toys and interesting games on one side of the barrier and a table and chair along with a desk bell on the other side of the barrier. The experimenter showed the child how to use the desk bell and informed them that once they left the room, the child could ring the bell and the experimenter would return. Upon returning (after the child rang the bell) the experimenter would reward the child and play a “game” with them. After a series of further instructions, the experimenter would then continue this interaction and assess the child's ability to “transform the reward objects that face him during the delay period in ways that either permit or prevent effective delay of gratification” (p. 259).

The final classification of the intervention type in the included studies was relaxation training interventions. Lakes and Hoyt's (2004) study was the most identifiable of this intervention type and involved periods of meditation where the children/adolescents were instructed to clear their minds of thoughts and worries while performing deep breathing techniques. Following this exercise, the subjects were then instructed to ask him/herself three questions intended to promote self-monitoring: 1) Where am I?; 2) What am I doing?; and 3) What should I be doing? After answering these questions the subjects were told to correct their thoughts and behavior if they were not consistent with the expectations of the particular situation. Ultimately, the instructors encouraged these exercises while emphasizing that the subject (not anyone else) is responsible for regulating their own behavior (p. 289).

Quality Assessment

It is important to note several methods for assessing the “quality” of the included studies. One of the most agreed upon determinants of study quality is the study's research design. Because all of the included studies were based on a randomized controlled experiment to evaluate the effectiveness of self-control improvement interventions, it is reasonable to assume that these studies are of high quality. Yet, it was rare for any of the studies to provide any detail on whether the randomization process was compromised or if attrition had any differential effects for the experimental/control groups. Thus, it is possible that some group imbalances might have arisen by chance. Further, most of the studies did not provide any information on whether the experimental/control groups were treated similarly throughout the course of the intervention by those who administered the intervention. Therefore, while we are confident that most studies were of sufficient quality because they used a research design involving random assignment, we still included a measure of whether there was substantial attrition reported in a particular study as a control measure (e.g., potential moderator) in the analysis that follows.