Abstract
BACKGROUND
The problem, condition or issue
Social and emotional competence is vital for forming and sustaining good relationships, solving everyday problems, adapting behaviour and making healthy choices throughout life. It includes self-awareness, understanding and working with others, controlling emotions and caring about oneself and others. Through social and emotional learning, children and adults are able to develop the skills necessary to work and live effectively in society, yet a significant proportion of pupils struggle with one or more facets of social and emotional development (Doll, Brehm, & Zucker, 2014; Jones & Bouffard, 2012; Zins, 2004). Estimates suggest that between 9.5 and 14.2 percent of children under five experience some form of social and emotional problem which negatively impact their functioning, development and school-readiness (Brauner & Stephens, 2006).
Underdeveloped social and emotional competencies can have consequences which are far reaching and long lasting (Daly, Delaney, Egan, & Baumeister, 2015; Durlak, 2015; Moffitt et al., 2011). Deficits in basic skills - such as the ability to identify emotions - can affect all stages of the lifespan, including: being rejected by others; exclusion from peer activities; being victimised; and lower peer-rated popularity (Lemerise & Arsenio, 2000; Leppänen & Hietanen, 2001; Mostow, Izard, Fine, & Trentacosta, 2002). Chronic physical aggression during primary school increases the risk of violence and delinquency through adolescence in boys (Broidy et al., 2003; Nagin & Tremblay, 1999), which in the long term can lead to destructive forms of emotion management, such as alcohol abuse. Poor self-regulation in childhood is associated with negative outcomes in adulthood including poorer health, social and economic outcomes (Compas, Connor-Smith, & Jaser, 2004; Daly et al., 2015; Kubzansky, Park, Peterson, Vokonas, & Sparrow, 2011; Moffitt et al., 2011), lack of (Nota, Soresi, & Zimmerman, 2004) and increased criminal behaviour (Henry, Caspi, Moffitt, Harrington, & Silva, 1999).
Social and emotional learning (SEL) programmes aim to intervene early to address skill deficits and equip children with the social and emotional competencies they need for life (Eisenberg, Spinrad, & Eggum, 2010; Moffitt et al., 2011). Attention has increasingly turned to facilitating social and emotional learning in the classroom as a means to help children develop skills such as; empathy, emotional regulation and behaviour management strategies, that will enable them to become functioning members of society in adulthood (Humphrey, 2013; Humphrey, Lendrum, & Wigelsworth, 2010).
What is social and emotional learning?
Social and emotional learning (SEL) has been defined in different ways. Definitions tend to encompass a range of competencies, concepts and areas for development. For example, the American Collaborative for Academic, Social and Emotional Learning (CASEL, 2015) have described and defined five core competencies: self-awareness; self-management; social awareness; relationship skills; and responsible decision making. Waters & Sroufe (1983) also describe these social and emotional competencies as being important in enabling children “to generate and coordinate flexible, adaptive responses to demands and to generate and capitalize on opportunities in the environment” (p80). In the UK, The Young Foundation (McNeil, Reeder, & Rich, 2012) have identified a core set of evidence-based social and emotional capabilities shown to be important throughout the lifespan, including: communication; confidence and agency; planning and problem solving; relationships and leadership; creativity; resilience and determination; and managing feelings. Whilst core competencies vary in their definitions and scope between authors and agencies, a consistent pattern emerges demonstrating the array and reach of social emotional learning and its related competencies.
The importance of social and emotional learning
It has been evidenced both nationally and internationally (Barry, Clarke, Jenkins, & Patel, 2013; Weare & Nind, 2011; Yoshikawa et al., 2015) that improving social and emotional learning allows children to connect with others and to begin to learn in a more effective way, thereby increasing their chances of success both in school and in life. Many countries have endeavoured to embed SEL into school ethos and culture. For example; in England personal and social development is addressed at the policy and practice level through the inclusion of building character and resilience as a priority by English Department of Education's (Department of Education, 2016); In America, CASEL work to support the integration of SEL in education through research, practice and policy; In Australia, SEL has been embedded in the national curriculum under personal and social capability (Collie, Martin, & Frydenberg, 2017).
There is a growing consensus in academic and policy circles regarding the importance of children's social and emotional development and its links to behavioural and health outcomes (Ciarrochi, Deane, & Anderson, 2002; NICE, 2008; Petrides, Frederickson, & Furnham, 2004). Children who fail to achieve developmental milestones associated with SEL may be at risk of failing to make meaningful relationships with their peers and with the school situation (Zins, 2004; Zins & Elias, 2007). Social and emotional outcomes are related to educational outcomes (Duckworth & Seligman, 2005), emotional wellbeing (Eisenberg et al., 2010) and general life trajectories (Daly et al., 2015; Moffitt et al., 2011). Educationally, a range of social and emotional factors have been found to have an impact on educational achievement (Banerjee, Weare, & Farr, 2014; Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011), including willingness to learn, openness to new experiences and one's ability to interact with peers and teachers in the school situation.
Given the increasing emphasis on SEL in both the educational research and policy arenas, there has been increased importance placed on SEL in schools and the need for facilitating optimal development of SE competence in every child. A variety of school-based programmes have been developed which aim to target, remediate and/or improve social and emotional skills in children, from the very beginning of their educational careers. Within the context of education and schooling, these programmes are particularly important for a number of reasons. Firstly, children do not learn in isolation, but rather construct meaning based on their life experiences (Vygotsky, 1980) and the relationships they have built can either facilitate or impede the learning process (Zachary, 2011; Zeidner, Matthews, & Roberts, 2012). Secondly, social and emotional skills are necessary antecedents for learning and constructing knowledge and can help ensure that children are ready, willing and able to learn. Thirdly, strong social and emotional skills enable children to work together, work with others and work alone in the school setting.
Moderating factors
There is evidence to suggest that certain groups of children are more likely to have difficulties in specific areas of social and emotional development. Children from socially disadvantaged communities and children who suffer maltreatment, are found to be less likely to develop social and emotional skills in line with their peers (Blair & Raver, 2012). Research on similarities and differences between social and emotional development in boys and girls is equivocal with some studies suggesting girls and boys develop in different ways (Lehmann, Denissen, Allemand, & Penke, 2013) while others conclude that these differences may be exaggerated (Else-Quest, Higgins, Allison, & Morton, 2012). Finally, age is likely to be a moderating factor. While early intervention has been shown to produce greater impacts this is not the case for all relevant outcomes (Sklad, Diekstra, Ritter, Ben, & Gravesteijn, 2012). Understanding the developmental trajectory of social and emotional skills and identifying key developmental stages may help identify the age at which interventions are likely to confer most benefit.
Description of the intervention
This systematic review will focus on curriculum based SEL interventions delivered in preschool or primary/elementary schools aimed at improving social and emotional skills among pupils. It will include any universal programme, delivered on a whole-class or school basis. The interventions' primary goal must be to improve social and emotional competence and be delivered in a pre-school/kindergarten or primary/elementary school setting as part of the normal school day.
The intervention must be structured, include a taught component and be delivered directly to children and involve the active participation of the child. The aims of the programme must be explicitly related to child gains, as opposed to those which focus solely on teacher competencies or school ethos. The intervention must also be a curriculum based social emotional learning program. Interventions may be delivered by the class teacher, other school personnel or non-school personnel. Interventions must be delivered for a minimum of one school term.
How the intervention might work
Whilst interventions are varied in terms of their scope and their delivery, they typically adopt either a preventative or remediative approach and may use a range of theoretical approaches, such as Bronfenbrenner's ecological theory or the achievement model of emotional literacy (Rivers & Brackett, 2010). Many SEL interventions in preschool and primary/elementary school are based upon an implicit or explicit understanding of how social and emotional skills develop among children at this age. Rarely, however, do interventions outline a logic model which clearly sets out how the intervention creates or facilitates change in the desired outcomes. To this end, most interventions encourage and help children recognise and identify emotions or undesirable behaviours. Children are then taught, usually through scene setting, discussion and/or modelling techniques, skills to help them reflect on the problem and think through (and thus modify) the various and alternative actions they might take when faced with a difficult or conflict-related situation.
A range of programmes have emerged that target children who are not meeting the developmental milestones associated with social and emotional learning. Some of these milestones include, for example: being able to understand complex and simultaneous feelings; consider multiple perspectives; empathise or sympathise with others; and recognise, regulate and express emotions effectively in order to create and sustain personal and social relationships. In so doing, SEL interventions aim to support children to enhance their positive emotions and behaviours, whilst also moderating their negative emotions and associated behaviours.
Programmes such as PATHS (Promoting Alternative Thinking Strategies) are implemented widely in a number of countries and are designed to facilitate the development of self-control, emotional awareness and interpersonal problem-solving skills. PATHS does this by teaching children strategies to help them regulate their emotional response to a difficult situation by: stopping to reflect on the problem, thinking through alternative solutions and based on this, choosing an appropriate course of action. Here, the programme's aims and objectives are closely tied with the developmental achievements which have been set out in previous literature. Similarly, Roots of Empathy – a classroom based SEL intervention - seeks to promote prosocial behaviour and reduce aggressive behaviour by enhancing children's emotional recognition, empathy and emotional regulation. It is a structured curriculum-based programme that involves a mother and baby coming to class. Children observe the baby's development over the period of a school year and are taught to ‘recognise’ the baby's feelings and reflect on their own and others feelings. Learning about the baby's development, how the baby might be feeling, why they might be upset or happy and observing the loving parent-child relationship provides children with a model of responsible parenting from which to learn. The intervention relies on a mixture of modelling, discussion and storytelling to teach children about their own and others' emotions and behaviours.
Based on the logic model suggested by CASEL, explicit instruction in SEL skills (provided through the intervention) combined with teacher instructional practices that are both integrated with the academic curriculum and school ethos, should lead to the acquisition of SEL skills, improved attitudes towards self, others and learning as well as an enhanced learning environment. In turn, these changes result in positive social behaviour, fewer conduct problems, lower emotional distress and improved academic performance.
For the purpose of the current review, the co-primary outcomes will be behavioural, namely: improved prosocial behaviour and fewer conduct problems. Secondary outcomes will include: SEL skills (including emotional regulation, emotional recognition, empathy, problem solving); attitudes (including self-esteem, peer relationships, enjoyment of school); emotional distress and academic performance.
Whilst certain background or demographic characteristics may moderate the impact of a programme on certain groups of children or families (previously discussed), other programme related factors – such as quality and fidelity of implementation – can also moderate effects. A number of programme factors emerge which could be seen to be important when implementing a programme. As mentioned in Clarke et al's (2015) recent review, parental involvement may be an area which facilitates success of an intervention. Here it was found that when parents were involved and knowledgeable about the process being undertaken children were more likely to make significant gains. Additional factors which may lead to greater successes include teachers and other school staff implementing the programme compared to instructors from outside the school (Durlak et al., 2011, p.13) which can allow children to make greater gains, perhaps because they feel calm and confident with the instructor. There is also some evidence to suggest that structured programmes which incorporate training for those delivering the program, are interactive in nature and which guide young people towards a specific set of goals are more likely to be successful (Smith, Schneider, Smith, & Ananiadou, 2004). Moreover, there is some evidence to suggest that multi-component programmes may have added benefits compared with single component programs (Adi, Killoran, Schrader McMillan, & Stewart-Brown, 2007; Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2004; Humphrey et al., 2010; Wells, Barlow, & Stewart-Brown, 2003). Factors regarding programme design and implementation are also influential, with programmes which were identified as well-designed and well implemented being the most impactful (Durlak et al., 2011). Stage of programme development may affect the success of the intervention, with efficacy trials conducted under controlled conditions and often led by the programme developer resulting in greater effects than effectiveness trials conducted under more ‘real world’ conditions (Wigelsworth et al., 2016).
These programme content and implementation factors will be documented in the coding framework and, if possible, analysed as moderators of intervention effect.
Why it is important to do the review
The importance of SEL is now well recognised, and there has been significant growth in the number and type of SEL programmes offered in schools. Existing related reviews either do not focus on broad-based SEL programs or are currently out of date, or have included studies which used non-robust methodologies.
The Campbell and Cochrane systematic review libraries were searched in December 2017 for completed and ongoing reviews relevant to this area. This search found a number of relevant ongoing or completed reviews:
A review of universal school-based social information processing interventions for aggressive behaviour (Wilson & Lipsey, 2007). This review only included studies published until 2003 and focussed mainly on violence reduction and prevention as opposed to broader social emotional learning A review of school-based programmes to reduce bullying and victimisation (Farrington & Ttofi, 2009). This review only considered evaluations which measured bullying or aggression towards peers but did not consider wider social and emotional skills. A review of self-control programmes for reducing delinquency and problem behaviours (Piquero, Jennings, Farrington, & Jennings, 2010). This review did not have an explicit focus on universal, school-based programmes (79% of included studies were school-based, and only a third were universal). A protocol for a review of school-based interventions for reducing disciplinary school exclusion (Valdebenito, Eisner, Farrington, Ttofi, & Sutherland, 2015). This review focuses on school exclusions and thus does not include a wider range of social emotional learning measures. A protocol for a review of school-based executive functioning interventions (Steenbergen-Hu, Olszewski-Kubilius, & Calvert, 2017). This review focusses solely on core components of executive function, and aims to use direct executive function outcomes and so does not focus on wider social and emotional learning. A review of mindfulness-based interventions for improving academic achievement, behaviour and socio-emotional functioning of primary and secondary students (Maynard, Solis, Miller, & Brendel, 2017). This review restricts its focus to interventions making use of mindfulness tools rather than including interventions that use a wider range of techniques to improve social and emotional learning. A protocol for a review of practices and programme components for enhancing prosocial behaviour in children and youth (Spivak, Lipsey, Farran, & Polanin, 2015). This review focusses on pro-social behaviour in the classroom setting, and does not have a specific focus on interventions aimed at the broader range of social and emotional outcomes. A review of effective programmes for social and emotional learning (Cocoran & Slavin, title registered January 2016). This review is mainly focused on academic attainment, and how social and emotional programmes impact attainment. This review does not have a focus on social and emotional outcomes. A review of the Tools of Mind curriculum for improving self-regulation in young children (Baron, Evangelou, Malmberg, & Melendez-Torres, 2017) focused on just one intervention.
In the wider research literature, several other reviews have been conducted in the area of social and emotional learning (SEL) programmes (Browne, Gafni, Roberts, Byrne, & Majumdar, 2004; Clarke, Hussein, Morreale, Field, & Barry, 2015; Durlak et al., 2011; Payton et al., 2008; Sklad et al., 2012; Taylor, Oberle, Durlak, & Weissberg, 2017; Wigelsworth et al., 2016; Wilson & Lipsey, 2007). The most relevant of these is Durlak et al.'s (2011) meta-analysis, which focused on school-based programmes and their impact on a number of pupil outcomes including: social and emotional skills and attitudes; positive social behaviour; conduct problems; emotional distress; and academic performance. Durlak et al.'s searches were conducted up until the end of 2007 and thus the findings are now dated. Whilst more recent, Clarke et al.'s (2015) review focussed only on literature published in the UK, did not conduct meta-analyses and studies were included which did not include control groups. More recently, (Wigelsworth et al., 2016) examined the impact of developer involvement, trial stage and transferability on universal SEL interventions. This review represents an important step in explicating the complex relationships between efficacy and effectiveness, programme fidelity and transferability. However, it was limited to studies reported in English between 1995–2013, excludes pre-school programmes and excluded studies not reporting means and standard deviations, which misses an opportunity to identify gaps and bias in reporting of conducted trials. Finally, Taylor and colleagues (Taylor et al., 2017) conducted a meta-analysis of follow-up effects of SEL programs including studies reported in English up to December 2014. This review provides an analysis of the longer term impacts of SEL programs but does not include recent studies or studies in a language other than English.
Since these reviews have been published, the review team is aware of a number of new evaluations conducted on interventions aimed at pupil emotional wellbeing and behaviour through universal social and emotional learning programmes. An up-to-date systematic review including more recently conducted studies is therefore needed. Additionally, whilst there are a number of ongoing or completed narrative reviews which focus on individual programmes, specific social and emotional outcomes or specific niches or facets of socio-emotional development, the reviews which have been (or are being) completed currently tend not to systematically review, map and assess the wide range of social and emotional learning programmes currently being used in schools for children aged 3–11. This proposed review will therefore be more wide-ranging and inclusive and will allow for the inclusion of more recent literature emerging in the field.
Finally, this review is seeking to address a broader set of questions regarding the overall impact of universal school-based SEL programmes and, within this, has a particular focus on comparing the effectiveness of different types of intervention, and for different subgroups, in order to determine whether there are any underpinning programme-specific components that are associated with greater effects. These objectives require a broad-based review that has not been attempted by any of the reviews listed above.
OBJECTIVES
The objectives of the review are:
To identify, appraise and synthesise evidence from randomized and cluster randomized trials on the effectiveness of universal, school- based social and emotional learning programmes for young children age 3–11 as compared to no intervention or inactive control, for improving children's social and emotional skills and behavior. To identify and classify the core components of programmes and to assess which of these components, if any, can be considered as ‘active ingredients’ in terms of their presence or absence being associated with differential programme effectiveness. To ascertain whether fidelity of delivery and the stage of the programme's development are associated with programme effectiveness.
To assess if the programmes are differentially effective for different subgroups in relation to age, gender and socio-economic background. To develop a set of clear and specific recommendations, based on the evidence reviewed, to guide the further development and delivery of universal school-based social and emotional learning programmes.
METHODOLOGY
Characteristics of the relevant studies
Given the broad definition of social and emotional learning and the wide variety of school-based interventions that now exist, studies will be varied in terms of their design, methodology, implementation, outcomes and measurement. Four representative studies that could potentially meet the inclusion criteria for the current review, are detailed below.
Domitrovich, Cortes and Greenberg (2007) report a cluster randomised controlled trial evaluation of the PATHS (Promoting Alternative THinking Strategies) curriculum (Kusché & Greenberg, 1994). This classroom-based curriculum is designed to reduce problem behaviour whilst promoting social competence. This wait list control trial involved twenty classrooms in Pennsylvania, ten randomly allocated to each of the control and intervention groups. The intervention was delivered to children of nursery age (between three and four) by a trained class teacher, who implemented weekly lessons (30) and extension activities across a nine month period
Snyder, Vuchinich, Acock, Washburn and Flay (2012) conducted a randomised controlled trial on Positive Action, a character development programme which aims to improve school quality, improve academics, student behaviour, and character and to mitigate problem behaviours. This universal, school wide programme includes a school-wide climate development component, teacher/staff training, a coordinator's manual, school counsellor's programme, and PA coordinator/committee guide; and family- and community-involvement programmes. The sequenced elementary curriculum consists of six units which are divided into 140, 15- to 20-minute lessons per grade, per academic year, provided by classroom teachers. This trial involved 20 racially/ethnically diverse schools and was conducted between 2002 and 2006. Teacher, parent and student data were collected in pre and post tests and additional, school-level archival data were used to examine programme effects at one year post-trial.
Mendelson et al., (2010) conducted a randomised trial assessing the feasibility, acceptability, and preliminary outcomes of a pilot school-based mindfulness and yoga intervention. An existing intervention was manualised and structured into sessions which ran for four days a week for twelve weeks. Two schools took part, with half of the children in each school being assigned to the intervention group, which consisted of around 50 children. Holen and colleagues (Holen, Waaktaar, Lervåg, & Ystgaard, 2012) evaluated Zippy's Friends, a universal school based programme that aims at strengthening children's coping skills, with 1,483 children in 35 schools aged between seven and eight. This manualised, structured programme is delivered in 24 weekly lessons and is based on six stories about three cartoon characters. The main objective of the programme is to prevent psychological problems by increasing children's coping repertoire and giving them various ways of coping with problems.
Criteria for including and excluding studies
Types of study designs
Randomised controlled trials or cluster randomised controlled trials will be included. We anticipate a large number of included studies and so can reasonably restrict our analysis to only the highest quality research designs.
Types of participants
Children attending preschool or primary/elementary schools. This population is typically between the ages of three and eleven years.
Where a study includes participants from both preschool or primary schools and secondary schools reasonable attempts will be made to extract or source the data for only the children attending preschool/primary/elementary school from published and unpublished documentation. If data us unavailable authors will be contacted to request summary statistics for only the participants of interest.
Studies which are located in special schools or which focus on children with identified special educational needs or social, emotional or behavioural difficulties will be excluded.
Types of interventions
Studies will be included if the intervention is a universal, school- or classroom-based programme which is delivered, via a set curriculum, to a whole class or whole school and which primarily aims to improve social and emotional competencies of children. The intervention must be a curriculum-based social emotional learning programme.
The intervention must be delivered directly to children and involve the active participation of the child. The aims of the programme must be explicitly related to child gains, as opposed to those which focus solely on teacher competencies or school ethos.
Interventions may be delivered by the class teacher, other school personnel or non-school personnel. Interventions must be delivered for a minimum of one school term.
Studies must include an inactive comparison condition that could include;
Studies with an inactive control compared to two or more intervention arms can be included and the sample size of the control group will be divided by the number of eligible intervention arms to avoid double counting control group participants in any meta-analysis.
The following types of studies will be excluded:
Studies in which the SEL programme was delivered in an after-school setting or without a school-based component (e.g. in youth clubs, summer clubs, sports or social clubs, or through parenting groups); Studies in which the SEL programme targets specific groups associated with the outcomes of interest (e.g. pupils with special educational needs); and Studies in which the programme is not delivered directly to children (e.g. interventions aimed at changing the behaviour or attitudes in teachers without any direct involvement of children). Studies in which the programme is not delivered according to a pre-specified curriculum or manual (e.g. school providing leisure facilities to pupils). Studies in which the programme is delivered as a one off or infrequent intervention.
Kratochwill et al. (Kratochwill, McDonald, Levin, Scalia, & Coover, 2009) represents an example of a study that is likely to be excluded. This study evaluated the Families and School Together (FAST) programme (McDonald et al., 1997). Here, children were recruited from schools and the programme delivered in the school setting. The programme aimed to improve a number of outcomes, including parental involvement, child behaviour and teacher perceptions of attainment. To do this it delivered eight weekly sessions where children and parents took part in games and activities together. Whilst this study does include some aspects of social and emotional development, the main focus was on parents who are working alongside their children. This is a multi-family, group intervention, rather than one which focusses on the child and their social and emotional development. As there is no direct taught element which is delivered to the child this study would not meet the inclusion criteria
Types of outcome measures
Prosocial – behaviour. Conduct problems.
SEL skills including emotional regulation, emotional recognition, empathy, problem solving. Attitudes towards self and others including self-esteem, peer relationships, enjoyment of school. Emotional distress, such as anxiety, depression social withdrawal. Educational attainment. Adverse effects.
In SEL trials it is rare that adverse effects are considered and so we will not select adverse outcomes a priori. Instead we intend to extract any data on adverse outcomes and to provide a narrative summary of potential adverse effects.
Outcome measures vary widely in terms of quality and validity. We will exclude treatment inherent measures as they risk overestimation of effect size (Slavin & Madden, 2008). The minimum standard will be that for any instruments used in included studies a full description of the scale and its scoring as well as associated reliability statistics are available.
Search strategy
In order to find all eligible studies the following data sources will be searched and consulted. This comprehensive search will include multiple electronic databases, research registers, grey literature sources, and reference lists of reviews and relevant studies. We will not restrict the study selection in terms of language, date or publication status. Each of the following databases and trial registries will be searched using the search strings set out in appendix 1:
British Education Index Education Abstracts (EBSCO) ERIC MEDLINE PsycINFO Web of Science/ Knowledge Database Science Citation Index Web of Science/ Knowledge Database Social Science Citation Index National Registry of Evidence-based Programs and Practices
Relevant reviews will be searched for in the following databases;
Database of Abstracts of Reviews of Effectiveness The Campbell Library Cochrane Collaboration Library Evidence for Policy Practice Information and Coordinating Centre (EPPI-Centre)
We will also search grey literature and the following databases and websites will be used:
CASEL Education Endowment Foundation Google Scholar – using a series of searches and screening the first 2 pages of results for each search ProQuest Dissertations and Theses WorldCAT OECD Education Library Opengrey World Health Organization
Reference lists of relevant reviews and included studies will be screened and forward citation searching of included studies will be carried out through Google Scholar. These web searches will be limited to the first 2 pages of search results. Prominent authors in the field will also be contacted. In addition, a final step towards the end of analysis, a manual search of the most recent issue(s) of key journals will be conducted. To do this, the top 10 journals that have provided included studies so far, will be identified and their most recent issues will be checked.
One reviewer will conduct the database searches, remove duplicates and obviously irrelevant records. We anticipate that the searches will result in a very large number of records to screen and so to ensure robustness, each report will be screened by title and abstract by two reviewers. Potentially eligible studies will then be retrieved in full text form and two reviewers will screen each full text. Any disagreements will be discussed with the wider review team until a consensus is reached.
Search terms and keywords
The search strategy has been developed using a modified version of the pearl harvesting approach. First, one author (JR) extracted keywords from ten randomly selected relevant studies. Each of the terms extracted were then searched individually within the thesauri in ERIC, British Education Index and Psycinfo and all additional terms added to the list of possible search terms relating to either: the participants; intervention setting; study design; or outcomes of interest. All authors then suggested additional terms that had not been identified (e.g. nursery school did not appear in any article of thesauri). Two authors (JR and JH) then screened the full list of terms and removed any duplicate, overlapping or irrelevant terms. The search string samples below therefore were generated by a combination of terms originating from the literature, terms originating from the review team and terms originating from a thesauri search.
A combination of four search strings will be used relating to: 1) participants; 2) the intervention setting; 3) the study design; and 4) the outcomes of interest. A sample search strategy is provided in appendix 1. Search strings and search limits will be modified to be suitable for each database. Search for exact phrases or proximity searching will be used to increase search specificity. In addition searches for a number of known named interventions will also be undertaken (see appendix 1).
Description of methods used in primary research
All included studies will be randomised or cluster-randomised controlled trials.
Criteria for determination of independent findings
It is important to ensure that the effects of an individual intervention are only counted once and the following conventions will therefore apply.
Where there are
Where the
Studies comparing
In the case of
Details of study coding categories
Once eligible studies have been found, an initial analysis of programme descriptions will be undertaken for all eligible programmes. This will be used to identify the core components of programmes and to develop an overarching typology and coding frame for these. From previous studies, and as referenced in the earlier discussions of existing literature, it is anticipated that such components are likely to include:
Duration and intensity of the programme. Whether the programme was delivered by external facilitators or internally by teachers or other school staff. Whether the programme included training and/or ongoing support for the facilitators. Whether the programme explicitly included parental involvement. How well structured the programme is. Whether the programme has a specific set of goals that it guides participants towards.
Alongside extracting data on programme components, descriptive information for each of the studies will be extracted and coded to allow for sensitivity and subgroup analysis. This will include information regarding:
the study characteristics in relation to: design, sample sizes, measures and attrition rates, who funded the study, and whether the study was conducted by a research team associated with the programme or an independent team. the stage of programme development, for example whether it is a new programme being piloted or an established programme being replicated. the level to which the programme was delivered with fidelity using information provided in the study report to categorise each programme as being delivered as with ‘high’, ‘medium’ or ‘low’ fidelity. the participants' characteristics in relation to age, gender and socio-economic background.
A coding framework has been devised and piloted (see appendix 2) and will be refined as necessary based on the above. Coding will be carried out by trained researchers. Each study will be coded by two members of the review team independently and discrepancies will be discussed and a consensus agreed.
Quantitative data will be extracted to allow for calculation of effect sizes (such as mean change scores and standard error or pre and post means and standard deviations). Data will be extracted for the intervention and control group on the relevant outcomes measured in order to assess the intervention effects. If study reports do not contain sufficient data to allow calculation of effect size estimates authors will be contacted to obtain necessary summary data, such as means and standard deviations or standard errors.
Assessment of methodological quality and potential for bias will be conducted using The Cochrane Risk of Bias tool. This is a standard tool, which takes the forms of a series of questions about the randomisation procedures and blinding. The overall quality of evidence relating to the primary outcomes will also be summarised using the GRADE system (Ryan & Hill, 2016).
Statistical procedures and conventions
Calculation of effect sizes
It is anticipated that most outcomes reported will be based upon continuous variables and so the main effect size metric to be used for the purposes of the meta-analyses will be the standardized mean difference, with its 95% confidence interval. Within this, Hedges' g will be used to correct for any small sample bias. Where other effect sizes have been reported, such as Cohen's d or risk ratios (for dichotomous outcomes) these will be converted to Hedges' g for the purposes of the meta-analysis using formulae provided in the Cochrane Handbook (Higgins & Green, 2011).
As most studies are likely to involve group-level allocation, where possible, data will be included which have been adjusted to account for the effects of clustering, typically through the use of multilevel modelling or adjusting estimates using the intra-cluster correlation coefficient (ICC). Where the effects of clustering have not been taken into account, estimates of effect size will be adjusted following guidance in the Cochrane Handbook. If ICC is not reported external estimates will be obtained from studies that provide the best match on outcome measures and types of clusters from existing databases of ICCs (Ukoumunne, Gulliford, Chinn, Sterne, & Burney, 1999) or other similar studies within the review.
Approach to meta-analysis
Given the diverse range of interventions that this review is likely to find, random effects models, using inverse-variance estimation, will be used as the basis for meta-analysis. The analysis will be conducted using Stata 14 and the range of commands externally developed to conduct meta-analysis with Stata such as metan and metareg (Palmer & Sterne, 2016). Heterogeneity will be measured and reported through the Q, I2 and Tau2 statistics.
Main effects and sensitivity analyses (Objective 1)
The main effects analysis, synthesising the evidence in relation to the effects of SEL programmes in general, will be undertaken using the approach to meta-analysis outlined above for each primary and secondary outcome in turn.
For each outcome, the following sensitivity analyses will also be undertaken to assess whether there are potential influences relating to:
Studies that appear to exert an undue influence on findings. Whether the study was undertaken by any of the review authors. Levels of attrition. Study quality (Studies with a “high” or “unclear” risk of bias on 3 or more of the 7 risk of bias domains in the Cochrane Risk of Bias assessment will be coded as low quality). Level of involvement of the program developer in the evaluation (led by the programme developer, developer involved but not leading, independent). Who funded the study (i.e. was the study funded by the programme developers or an independent funder). Stage of the programme's development (efficacy, effectiveness).
In relation to studies that appear to exert an undue influence, a further meta-analysis will be conducted for each outcome that omits these studies to assess whether their inclusion exerts an influence on the findings.
For the remaining six potentially influencing factors, a meta-regression will be conducted for each outcome. In each case, meta-regression will only be used if there are the minimum prerequisite of ten studies (Borenstein, Hedges, Higgins, & Rothstein, 2009). The independent variables added to the regression model in each case will consist of dummy variables associated with the potential influences. The statistical significance of the coefficients in the resultant models for each of the dummy variables will be used to assess, formally, whether there is a notable influence on the findings with regarding to the above factors. Overall conclusions regarding the potential influences of these factors will be drawn from an assessment of the consistency of the findings of the meta-regressions that were able to be conducted across the outcomes.
In addition, for each outcome, an assessment of whether there is any evidence of publication bias will be conducted by using the funnel-plot technique to visually display the distribution of un-weighted contrast effect sizes by sample size. We will apply Egger's regression asymmetry test to funnel plots to test for funnel plot asymmetry (Egger, Smith, Schneider, & Minder, 1997).
Assessment of core programme components (Objective 2)
For all the programme components, to be identified through the analysis of programme descriptions, dummy variables will be created to indicate the presence or absence of each. For each outcome, where there are sufficient studies, a meta-regression will be run with all of these dummy variables included. As before, and in each case, meta-regression will only be used if there are the minimum prerequisite of ten studies (Borenstein et al., 2009). The statistical significance of the estimated coefficient for each dummy variable will be used to test formally whether the associated component can be considered to be an ‘active ingredient’ or not. Standardised betas will then be used to assess the relative importance of each of these ‘active ingredients’.
Assessment of potential influence of programme fidelity (Objective 3)
For each outcome, where there are sufficient studies, a meta regression will be run that will include the measure of programme fidelity added as an independent variable. The statistical significance of the estimated coefficient(s) for this independent variable will be used to test formally whether programme fidelity has an influence on the effectiveness of SEL programmes. If the estimated coefficient is found to be statistically significant, it will then be used to illustrate the size of the influence of programme fidelity.
Assessment of differential effectiveness in relation to age, gender and socio-economic background (Objective 4)
Eligible studies will be coded in terms of:
whether the age of the participants falls into preschool (typically aged 3–5), lower elementary/primary school (typically aged 5–8) or upper elementary/primary school (typically aged 8–11). the effects for boys and girls where reported separately. In such cases, each study will generate two studies for the purposes of the subgroup analyses to follow (one based upon the sample for boys only, and the other for girls only). whether the participants are largely from economically deprived/disadvantaged neighbourhoods or not.
Three subgroup analyses will then be conducted in relation to each of the three factors above (age, gender and socio-economic background) for each of the primary and secondary outcomes. The subgroup analyses (based upon random-effects models), will group studies by sub-category and estimate overall effects sizes for each.
Treatment of qualitative research
This systematic review is limited to synthesising the available evidence on the effectiveness and of universal school-based social and emotional learning programmes. It is beyond the remit of this present review to attempt to also synthesise the associated evidence related to process evaluations of such programmes. As such, there is no plan to include qualitative research.
Footnotes
REVIEW AUTHORS
|
|
|
| Title: | Professor |
| Affiliation: |
Dean of Research, Faculty of Arts, Humanities and Social Sciences Director, Campbell Centre UK & Ireland Professor, School of Social Sciences Education and Social Work. |
| Address: | Queen's University Belfast, University Road |
| City: | Belfast |
| Postal Code: | BT7 1NN |
| Country: | Northern Ireland |
| Phone: | +44 (0)28 9097 3991 |
| Email: |
|
| Co-author: | |
| Name: |
|
| Title: | Dr |
| Affiliation: |
Deputy director, Campbell Centre UK & Ireland Centre for Evidence and Social Innovation Senior lecturer, School of Social Sciences Education and Social Work. |
| Address: | Queen's University Belfast, University Road |
| City: | Belfast |
| Postal Code: | BT7 1NN |
| Country: | Northern Ireland |
| Phone: | +44 (0)28 9097 5944 |
| Email: | |
| Co-author: | |
| Name: |
|
| Title: | Dr |
| Affiliation: |
Campbell Centre UK & Ireland Centre for Evidence and Social Innovation School of Social Sciences Education and Social Work. |
| Address: | Queen's University Belfast, University Road |
| City: | Belfast |
| Postal Code: | BT7 1NN |
| Country: | Northern Ireland |
| Phone: | 028 9097 2593 |
| Email: |
|
| Co-author: | |
| Name: |
|
| Title: | Dr |
| Affiliation: | Lecturer, Queen's University Belfast, University Road |
| Address: | Belfast |
| City: | BT7 1NN |
| Postal Code: | Northern Ireland |
| Country: | Queen's University Belfast, University Road |
| Phone: | 02890973177 |
| Email: |
|
| Co-author: | |
| Name: |
|
| Title: | Dr |
| Affiliation: | Lecturer, University College Dublin |
| Address: | School Of Education, Roebuck Offices, Belfield |
| City: | Dublin |
| Postal Code: | Dublin 4 |
| Country: | Ireland |
| Phone: | +353 1 7167987 |
| Email: |
|
ADVISORY GROUP
Members of the current Review Team (Connolly, Miller and Sloan) have been part of a team that has undertaken a large cluster-randomised controlled trial of the Roots of Empathy social and emotional learning programme in Northern Ireland. This trial has been funded by the National Institute for Health Research (ISRCTN07540423). A Roots of Empathy Regional Planning Group has been established to oversee the delivery of the programme. The Group is led by the Public Health Agency and includes representatives from each of the five Health and Social Care Trusts in Northern Ireland charged with the delivery of the programme.
To supplement the findings of the trial, and to inform future policy decisions regarding the promotion of social and emotional learning in schools, the Public Health Agency has commissioned Connolly and Miller to undertake this present systematic review. The Regional Planning Group has agreed to act as an Advisory Group for the review. They have considered and provided feedback on this protocol. The one change to this review that has been made as a result of this feedback has been to add a further objective to include an appraisal and synthesis of the evidence on the cost-effectiveness of social and emotional learning programmes.
The subsequent role of the Advisory Group will be to help interpret the findings of the systematic review and proposed meta-analysis, particularly in relation to identifying the implications for policy and practice.
ROLES AND RESPONSIBILITIES
Content: Connolly, Hanratty, Miller, Roberts, Sloan,. Systematic review methods: Hanratty, Miller. Statistical analysis: Connolly, Hanratty, Miller. Information retrieval: Hanratty, Sloan.
FUNDING
Review authors Connolly and Miller were awarded funding from the Public Health Agency (Northern Ireland) to conduct this review.
POTENTIAL CONFLICTS OF INTEREST
None of the review authors has a financial interest in this review. None of them has been involved in the development of interventions to improve social and emotional learning. Some of the authors (Connolly, Miller, Hanratty, Sloan) have either completed and/or are currently running trials of interventions that may fall within the scope of this present review. Connolly is currently supervising a PhD student whose doctoral research involves the development and pilot evaluation of a preschool social and emotional learning programme that may fall within the scope of this present review. Hanratty is currently conducting a Cochrane systematic review of child-focused interventions for anger and aggression in young children.
PRELIMINARY TIMEFRAME
Date you plan to submit a draft review: 30 June 2018.
PLANS FOR UPDATING THE REVIEW
Review authors will update the review five years after the submission date.