PROTOCOL: The FRIENDS preventive programme for reducing anxiety symptoms in children and adolescents: A systematic review

Policy relevance

Concerns and worries are a normal part of the everyday life of children and adolescents (Weems & Stickle, 2005). Childhood fears such as worries about loss or separation from parents or worries about personal injury, death and natural disasters are often a normal part of childhood development (Warren & Sroufe, 2004). However, as Warren and Sroufe (2004) argue, these worries can become problematic if they become persistent, frequent and severe enough to interfere with or limit the child's everyday life and functioning. There are therefore good reasons for increased focus on anxiety prevention over treatment.

Evidence shows that a high proportion of children do not grow out of their anxiety disorders during adolescence and adulthood (Majcher & Pollack, 1996). Anxiety disorders are amongst the most prevalent psychiatric disorders, with a prevalence rate of 10.4% in Western European, North American and Australasian populations (Baxter et al., 2012). According to research by Thompson et al. (2004, 2008) most sufferers of anxiety do not access treatment until well into adulthood and even those who do access appropriate help typically suffer for many years before receiving that help. It is therefore important to prevent that children and youth with elevated anxiety symptoms move on to fully develop anxiety disorders.

Other reviews

We have located three systematic reviews on anxiety preventive programmes for children and adolescents in general (including FRIENDS) and three reviews on the specific programme FRIENDS.

Werner-Seidler et al. (2017) and the update Werner-Seidler et al. (2021):

Performed a systematic review on school-based depression and anxiety prevention programmes for young people (i.e., children or adolescents with a mean age between 5 and 19 years met their inclusion criteria). In the update the authors searched up to October 2020. Of the 118 included studies, 34 studies were classified as focused on preventing anxiety, and 30 were classified as being mixed depression/anxiety prevention programmes. A total of 18 evaluations of the FRIENDS anxiety prevention programme were included (and 4 follow-up studies), the majority (15) were classified as anxiety prevention programmes, but 3 were classified as mixed depression/anxiety prevention programmes. The classification of the focus of the studies, however, plays no role in the analyses as all studies reporting anxiety outcomes (72) were used in the analyses of anxiety regardless of the focus of the study as classified by the review authors. Separate meta-analyses were performed for depression and anxiety outcomes. Single factor subgroup analyses and multiple meta-regressions were performed with a number of study characteristics entered as predictors of the outcomes. In particular programme content was examined, but the impact of FRIENDS was not investigated, rather the impact of CBT-based programmes vs other therapeutic approaches was investigated. None of the effect sizes analysed were corrected for clustering at neither the school nor the class level (if needed). As the review included both universal prevention programmes, which typically are both assigned and delivered at the class or school level and targeted (selective/indicative) prevention programmes which may be assigned at the school or class level but is not delivered to whole classes, this may have biased the results of the meta-analyses considerably.

Ahlen et al. (2015):

Performed a systematic review on universal prevention programmes targeting anxiety or/and depression in school-aged children 6–18 years. The searches were performed in July 2012. Of the 30 included studies, the primary aim of the intervention was to prevent depression in 13 studies, to prevent anxiety in 10 studies, and to prevent both anxiety and depression in 7.

A total of six evaluations of the FRIENDS anxiety prevention programme were included (and two follow-up studies), all were classified as anxiety prevention programmes.

All studies reporting anxiety outcomes (18) were used in the analyses of anxiety regardless of the primary aim of the intervention as classified by the review authors. Separate meta-analyses were performed for depression and anxiety outcomes. The intervention ‘FRIENDS for life’ was examined as moderator for anxiety symptoms using single factor subgroups. No significant differences were found.

Higgins and O'Sullivan (2015).

The review provided a narrative summary of five randomised controlled trials (and two follow-up studies) which examined the effectiveness of the FRIENDS programme as a preventative universal intervention for child and youth (aged 4–16 years) anxiety. Studies published in peer-reviewed journals between 2000 and 2013 were eligible.

Maggin and Johnson (2014).

This is a systematic review with meta-analysis on school based FRIENDS for students enroled in Kindergarten to grade 12. Only group based experimental or quasi experimental studies with a control group reporting on standardised measures of anxiety were eligible. The review included 17 studies (reported in 16 manuscripts) of which 2 were follow-up studies.

The final search date (reported in Maggin & Johnson, 2019, which was a reply to a critique provided in Barrett et al., 2017) was October 2010. All analyses in the review were separated by the review authors' definition of low-risk students and students with elevated risk (defined as pre-test scores within the clinical range, though the cut-off scores used were not reported).

There are further a number of other ambiguities. First, it is unclear which studies provide effect estimates to which analyses (low/high risk students and post/follow up) as the numbers available (according to table 1) do not add up to the numbers reported used in table 4. Second, it is unclear which anxiety measures are used. Five studies reported results for two or more standardised anxiety measures. The review authors stated that they randomly selected one measure from each of these studies to include in the meta-analyses. It was not reported which ones were used. Last, it was reported that ‘a series of moderator analyses of student characteristics and programme features failed to predict treatment outcomes’ (p. 295), however, the analyses were not shown, and it was not reported which student characteristics and programme features were used nor which model was used other than ‘a mixed effects framework’ was used.

Fisak et al. (2011).

The purpose of this review was to provide a comprehensive review of the effectiveness of child and adolescent (below the age of 18) anxiety prevention programmes, including universal as well as targeted (selective/indicative) programmes. Programmes in which depression or general stress management was the primary goal, and where anxiety was only measured as a secondary variable, were not eligible. The date of database search is not reported, but it is reported that the hand searches (of selected journals) were performed from 1970 to the end of 2009. A total of 30 studies (of which 4 were follow-up studies to previously published studies) with a comparison group was found of which 10 were evaluating the FRIENDS intervention. Based on moderator analyses, it was found that studies utilising the FRIENDS programme were more effective than programmes not utilising FRIENDS. However, as noted by the authors, more research is needed to determine the degree to which the effectiveness of the programme is generalisable to nations other than Australia (8 of the 10 evaluations on FRIENDS were performed in Australia). None of the effect sizes analysed were corrected for clustering at neither the school nor the class level (if needed). As the review included both universal prevention programmes and targeted (selective/indicative) prevention programmes this may have biased the results of the meta-analyses considerably.

Briesch et al. (2010).

This was a report on a literature search, conducted to identify all empirical studies of the FRIENDS programme published in peer-reviewed journals. The search was not described nor documented other than that they screened the list of research abstracts provided on the programme developers' website (no web address was reported), and conducted literature searches using the PsycINFO and MEDLINE databases. No search terms or date was reported. The review cannot be labelled systematic and further no meta-analysis was performed. The authors reported the range of effect sizes in the studies (14 studies were found) and an average (simple average) effect size but no other statistics such as standard errors, confidence intervals or p-values.

We specifically searched the Cochrane Library for Cochrane systematic reviews and located one marginally relevant for the current review. James et al. (2005) and the updates James et al. (2015) and James et al. (2020), examined the effect of CBT treatment interventions for childhood anxiety disorders. Eligible participants were children and adolescents younger than age 19, who met diagnostic criteria for an anxiety disorder diagnosis. Primary outcomes were remission of primary anxiety diagnosis post-treatment and number of participants lost to post-treatment assessment. Secondary outcomes included remission of all anxiety diagnoses, reduction in anxiety symptoms and depressive symptoms and improvements in global functioning. In the most updated (James et al., 2020) 87 studies were included of which 5 evaluated FRIENDS. No separate analysis of FRIENDS was provided nor was FRIENDS included as a moderator.

Besides being up-to-date, a major difference between these systematic reviews and the current review is that we will focus on the FRIENDS intervention delivered as universal, selective and indicated preventive programmes. We will only include studies with a control group and with participants from at least two units (e.g., school or class) in each of the groups (treatment and control). All relevant outcome areas will be analysed separately in a meta-analysis taking into consideration the unit of analysis (cluster or individual) and the dependencies between effect sizes. We will be transparent concerning which studies and which measures are used in each analysis. In addition, the specific data used for any cluster correction and any moderator analyses will be reported in detail.

Types of studies

The proposed project will follow standard procedures for conducting systematic reviews using meta-analysis techniques.

Randomised (and non-randomised) controlled trials will be included. To summarise what is known about the possible causal effects of programme participation, we will include all study designs that use a control group, that is, a group of children/youth not participating in the intervention. The control group may be offered no treatment or treatment as usual.

The study designs we will include in the review are: 1.

Controlled trials (where all parts of the study are prospective, such as identification of participants, assessment of baseline, and allocation to intervention, and which may be randomised or non-randomised), assessment of outcomes and generation of hypotheses (Higgins & Green, 2011).

Studies using single group pre-post comparisons will not be included. Non-randomised studies using an instrumental variable approach will not be included—see Supporting Information: Appendix 1 (Justification of exclusion of studies using an instrumental variable (IV) approach) for our rationale for excluding studies of these designs. A further requirement to all types of studies (randomised as well as non-randomised) is that they are able to identify an intervention effect. Studies where, for example, the treatment is offered to children in one unit (e.g., school or class) only and the comparison group is children at another unit (school/class or more schools/classes for that matter) cannot separate the treatment effect from the unit of school/class effect.

Types of participants

The review will include children and adolescents aged 4 to 16 years who do not meet diagnostic criteria for an anxiety disorder diagnosis. We anticipate that the three age-appropriate programmes will be provided to children and adolescents in the corresponding age groups; that is, ‘Fun Friends’ to 4–7 year olds, ‘Friends for Life’ to 8–11 year olds and ‘My Friends Youth’ to 12–16 year olds. If studies including participants out-of-age range are located they will be eligible if at least 70% of participants are within the age range corresponding to the particular programme or results for a discrete age group within the eligible range is provided. If studies include a mix of children and adolescents with and without a clinically diagnosed anxiety disorder, we will include them if at least 70% of participants are not diagnosed or results of the eligible subgroup (not diagnosed) is provided.

Types of interventions

The intervention of interest is the preventive anxiety programme FRIENDS. Prevention is defined as those interventions that occur before the onset of a clinically diagnosed disorder. Treatment programmes (for those with a diagnosis) are not eligible. The type of prevention may be universal (applied to the general population, without focusing on the risk status), selective (target those (individuals or groups) who are identified as exhibiting an elevated risk for developing a disorder based on established risk factors) or indicated (target those (individuals) who exhibit problematic behaviours predicting a high level of risk or initial symptoms of a disorder but who do not yet meet criteria for the disorder).

The three age-appropriate preventive anxiety programmes: Fun FRIENDS, FRIENDS for Life (titled FRIENDS for Children before 2005), and My FRIENDS Youth (titled FRIENDS for Youth before 2005) are eligible.

The comparison population are children and adolescent who do not participate in any of the FRIENDS programmes.

Types of outcome measures

The intervention is an anxiety prevention programme and although some studies may report depression outcomes we will limit the analysis to anxiety outcomes. The reason is that an analysis on depression outcomes may be biased as not all studies report depression outcomes, and it cannot be ruled out that those who do have results biased towards a positive effect on depression outcomes.

Duration of follow-up

Time points for measures considered will be:

post intervention (including booster sessions)

Types of settings

All types of settings (e.g., school based, community based) will be eligible.

Electronic searches

The following electronic bibliographic databases will be searched:

ERIC (EBSCO)

We have consulted the list of databases comprised in the article by Kugley (2017) (Kugley, S., Wade, A., Thomas, J., Mahood, Q., Jørgensen, A.-M.K., Hammerstrøm, K. and Sathe, N. [2017]. Searching for studies: A guide to information retrieval for Campbell systematic reviews. Campbell Systematic Reviews, 13, 1–73. https://doi.org/10.4073/cmg.2016.1).

Searching other resources

Description of methods used in primary research

Randomised controlled trials are eligible, and we expect that most studies will be conducted with randomisation of participants. Studies conducted without randomisation of participants are eligible but are required to have a control group for inclusion in the review. Participants may be allocated by, for example, location differences, decision makers, policy rules or participant preferences. We expect that the primary studies demonstrate pretreatment group equivalence via matching, statistical controls, or evidence of equivalence on key risk variables and participant characteristics as outlined in the Section 3.3.4.

Selection of studies

Under the supervision of review authors, two review team assistants will first independently screen titles and abstracts to exclude studies that are clearly irrelevant. Studies considered eligible by at least one assistant or studies were there is insufficient information in the title and abstract to judge eligibility, will be retrieved in full text. The full texts will then be screened independently by two review team assistants under the supervision of the review authors. Any disagreement of eligibility will be resolved by the review authors. Exclusion reasons for studies that otherwise might be expected to be eligible will be documented and presented in an appendix.

The study inclusion criteria will be piloted by the review authors and team assistants (see Supporting Information: Appendix 2). The overall search and screening process will be illustrated in a flow diagram. None of the review team members will be blind to the authors, institutions, or the journals responsible for the publication of the articles.

Data extraction and management

Two review authors will independently code and extract data from all included studies. A coding sheet will be piloted on several studies and revised as necessary (see Supporting Information: Appendix 3). Disagreements will be resolved by consulting a third review author with extensive content and methods expertise. Disagreements resolved by a third reviewer will be reported. Data and information will be extracted on: available characteristics of participants, intervention characteristics and control conditions, research design, sample size, risk of bias and potential confounding factors, outcomes, and results. Extracted data will be stored electronically. Analysis will be conducted using RevMan5 and Stata software.

Assessment of risk of bias in included studies

We will assess the risk of bias in randomised studies using Cochrane's revised risk of bias tool, ROB 2 (Higgins et al., 2019).

The tool is structured into five domains, each with a set of signalling questions to be answered for a specific outcome. The five domains cover all types of bias that can affect results of randomised trials.

The five domains for individually randomised trials are: (1)

bias arising from the randomisation process;

For cluster-randomised trials, an additional domain is included ([1b] Bias arising from identification or recruitment of individual participants within clusters). We will use the latest template for completion (currently it is the version of 15 March 2019 for individually randomised parallel-group trials and 20 October 2016 for cluster randomised parallel-group trials). In the cluster randomised template however, only the risk of bias due to deviation from the intended intervention (effect of assignment to intervention; intention to treat ITT) is present and the signalling question concerning the appropriateness of the analysis used to estimate the effect is missing. Therefore, for cluster randomised trials we will only use the signalling questions concerning the bias arising from identification or recruitment of individual participants within clusters from the template for cluster randomised parallel-group trials; otherwise we will use the template and signalling questions for individually randomised parallel-group trials.

We will assess the risk of bias in non-randomised studies, using the model ROBINS –I, developed by members of the Cochrane Bias Methods Group and the Cochrane Non-Randomised Studies Methods Group (Sterne et al., 2016a). We will use the latest template for completion (currently it is the version of 19 September 2016).

The ROBINS-I tool is based on the Cochrane RoB tool for randomised trials, which was launched in 2008 and modified in 2011 (Higgins et al., 2011).

The ROBINS-I tool covers seven domains (each with a set of signalling questions to be answered for a specific outcome) through which bias might be introduced into non-randomised studies: (1)

bias due to confounding;

The first two domains address issues before the start of the interventions and the third domain addresses classification of the interventions themselves. The last four domains address issues after the start of interventions and there is substantial overlap for these four domains between bias in randomised studies and bias in non-randomised studies trials (although signalling questions are somewhat different in several places, see Sterne et al., 2016b and Higgins et al., 2019).

Randomised study outcomes are rated on a ‘Low/Some concerns/High’ scale on each domain; whereas non-randomised study outcomes are rated on a ‘Low/Moderate/Serious/Critical/No Information’ scale on each domain. The level ‘Critical’ means: the study (outcome) is too problematic in this domain to provide any useful evidence on the effects of intervention, and it is excluded from the data synthesis. The same critical level of risk of bias (excluding the result from the data synthesis) is not directly present in the RoB 2 tool, according to the guidance to the tool (Higgins et al., 2019).

In the case of an RCT, where there is evidence that the randomisation has gone wrong or is no longer valid, we will assess the risk of bias of the outcome measures using ROBINS-I instead of ROB 2. Examples of reasons for assessing RCTs using the ROBINS-I tool may include studies showing large and systematic differences between treatment conditions while not explaining the randomisation procedure adequately suggesting that there was a problem with the randomisation process; studies with large scale differential attrition between conditions in the sample used to estimate the effects; or studies selectively reporting results for some part of the sample or for only some of the measured outcomes. In such cases, differences between the treatment and control conditions are likely systematically related to other factors than the intervention and the random assignment is, on its own, unlikely to produce unbiased estimates of the intervention effects. Therefore, as ROBINS-I allow for an assessment of for example confounding, we believe it is more appropriate to assess effect sizes from studies with a compromised randomisation using ROBINS-I than ROB 2. If so, we will report this decision as part of the risk of bias assessment of the outcome measure in question. As other effect sizes assessed with ROBINS-I, these effect sizes may receive a ‘Critical’ rating and thus be excluded from the data synthesis.

We will stop the assessment of a non-randomised study outcome as soon as one domain in the ROBINS-I is judged as ‘Critical’.

‘Serious’ risk of bias in multiple domains in the ROBINS-I assessment tool may lead to a decision of an overall judgement of ‘Critical’ risk of bias for that outcome, and it will be excluded from the data synthesis.

Measures of treatment effect

Unit of analysis issues

Errors in statistical analysis can occur when the unit of allocation differs from the unit of analysis. In cluster randomised trials, participants are randomised to treatment and control groups in clusters, either when data from multiple participants in a setting are included (creating a cluster within school or community setting), or when participants are randomised by treatment locality or school. Non-randomised studies may also include clustered assignment of treatment. Effect sizes and standard errors from such studies may be biased if the unit-of-analysis is the individual and an appropriate cluster adjustment is not used (Higgins & Green, 2011).

A study design where participants are individually allocated to treatment, but the treatment is delivered in a group setting, are known as individually randomised group treatment trials (Pals et al., 2008). The analysis in such a study design must also correct for the fact that dependencies may arise between individuals that happen to receive the intervention in the same group.

If possible, we will adjust effect sizes individually using the methods suggested by Hedges (2007b) and information about the intra-cluster correlation coefficient (ICC), realised cluster sizes, and/or estimates of the within and between variances of clusters. If it is not possible to obtain this information, we will adjust effect sizes using estimates from the literature (we will search for estimates of relevant ICC's), and assume equal cluster sizes. To calculate an average cluster size, we will divide the total sample size in a study by the number of clusters.

We will perform analyses separated by the time points suggested in Section 3.1.5.

Criteria for determination of independent findings

To determine the independence of results in included studies, we will consider whether individuals may have undergone multiple interventions, whether there were multiple treatment groups, whether several studies are based on the same data source and whether studies report multiple conceptually similar outcomes.

Dealing with missing data

Missing data and attrition rates will be assessed in the included studies; see Section 3.3.4. Where studies have missing summary data, such as missing standard deviations, the review authors will request this information from the principal investigators. If no information is yielded, we will derive these where possible from F-ratios, t-values, chi-squared values and correlation coefficients using the methods suggested by Lipsey and Wilson (2001). If missing summary data cannot be derived, the study results will be reported in as much detail as possible.

Assessment of heterogeneity

Heterogeneity amongst primary outcome studies will be assessed with chi-squared (Q) test, and the I-squared, and τ-squared statistics (Higgins et al., 2003). Any interpretation of the chi-squared test will be made cautiously on account of its low statistical power.

Assessment of reporting biases

Reporting bias refers to both publication bias and selective reporting of outcome data and results. Here, we state how we will assess publication bias.

We will use funnel plots for information about possible publication bias if we find sufficient studies (Higgins & Green, 2011). However, asymmetric funnel plots are not necessarily caused by publication bias (and publication bias does not necessarily cause asymmetry in a funnel plot). In general, asymmetry is a sign of small-study effects, of which there can be many causes beside publication bias (Sterne et al., 2005).

Instead of trying to interpret the funnel plots as direct evidence of publication bias, or the lack thereof, we will perform sensitivity analyses for publication bias in meta-analyses as suggested by Mathur and VanderWeele (2020). This method gives a value of how large ratios of publication probabilities (i.e., the likelihood of affirmative results to be published relative to non-affirmative results) would have to be to alter the results and therefore indicate how robust the meta-analysis is to publication bias.

Data synthesis

The proposed project will follow standard procedures for conducting systematic reviews using meta-analysis techniques.

All follow-up durations reported in the primary studies will be recorded, and we will do separate analyses for post, short-term, medium term and long-term outcomes.

The overall data synthesis will be conducted where effect sizes are available or can be calculated, and where studies are similar in terms of the outcome measured. Meta-analysis of outcomes will be conducted on each metric (as outlined in Section 3.1.4) separately.

As different computational methods may produce effect sizes that are not comparable, we will be transparent about all methods used in the primary studies (research design and statistical analysis strategies) and use caution when synthesising effect sizes. Special caution will be taken concerning studies using regression discontinuity designs (RDD) to estimate the treatment effect. In sharp RDDs, a threshold of a (non-manipulable) forcing/running variable determines which students receive a treatment and which do not, that is, the design is similar to a RCT in the sense that the random sequence determining treatment assignment can be seen as a running variable (Lee & Lemieux, 2010). In contrast, in ‘fuzzy’ RDDs, being on one side of a threshold is a special type of IV only makes it more likely that a student ends up in the treatment or control group, and the threshold is used as an instrument to estimate local average treatment effects (Angrist & Pischke, 2009; Imbens & Lemieux, 2008). That is, fuzzy RDD is a form of IV analysis, which we will exclude due to the comparability issues mentioned earlier. Sharp RDDs will be included, but, as the effects may be estimated on a very ‘local’ sample close to a threshold, may be subject to a separate analysis depending on the comparability to samples from other studies. We will in any case check the sensitivity of our results to the inclusion of RDD studies. In addition, we will discuss the limitation in generalisation of results obtained from these types of studies.

When the effect sizes used in the data synthesis are odds ratios, they will be log transformed before being analysed. The reason is that ratio summary statistics all have the common feature that the lowest value that they can take is 0, that the value 1 corresponds with no intervention effect, and the highest value that an odds ratio can ever take is infinity. This number scale is not symmetric. The log transformation makes the scale symmetric: the log of 0 is minus infinity, the log of 1 is zero, and the log of infinity is infinity.

Studies that have been coded with a Critical risk of bias will not be included in the data synthesis.

As the intervention deals with diverse populations of participants (from different countries, facing different life circumstances, etc.), and we therefore expect heterogeneity amongst primary study outcomes, all analyses of the overall effect will be inverse variance weighted using random effects statistical models that incorporate both the sampling variance and between study variance components into the study level weights. Random effects weighted mean effect sizes will be calculated using 95% confidence intervals, and we will provide a graphical display (forest plot) of effect sizes. Graphical displays for meta-analysis performed on ratio scales sometimes use a log scale, as the confidence intervals then appear symmetric. This is however not the case for the software Revman 5 which we plan to use in this review (If we apply robust variance estimation [RVE], the analysis will be conducted in Stata or R as RVE is not implemented in Revman 5). The graphical displays using odds ratios and the mean effect size will be reported as an odds ratio. Heterogeneity amongst primary outcome studies will be assessed with chi-squared (Q) test, and the I-squared, and τ-squared statistics (Higgins et al., 2003). Any interpretation of the chi-squared test will be made cautiously on account of its low statistical power.

In addition to 95% confidence intervals we will report 95% prediction intervals.

For subsequent analyses of moderator variables that may contribute to systematic variations, we will use the mixed-effects regression model if there are a sufficient number of studies. This model is appropriate if a predictor explaining some between-studies variation is available, but there is a need to account for the remaining uncertainty (Hedges & Piggott, 2004; Konstantopoulos, 2006).

Studies may provide results separated by, for example, age and/or gender. We will include results for all age and gender groups. To take into account the dependence between such multiple effect sizes from the same study, we will apply RVE approach (Hedges et al., 2010). An important feature of this analysis is that the results are valid regardless of the weights used. For efficiency purposes, we will calculate the weights using a method proposed by Hedges et al. (2010). This method assumes a simple random-effects model in which study average effect sizes vary across studies (τ ²) and the effect sizes within each study are equi correlated (ρ). The method is approximately efficient, since it uses approximate inverse-variance weights: they are approximate given that ρ is, in fact, unknown and the correlation structure may be more complex. We will calculate weights using estimates of τ ², setting ρ = 0.80 and conduct sensitivity tests using a variety of ρ values; to assess if the general results and estimates of the heterogeneity is robust to the choice of ρ. We will use the small sample adjustment to the residuals used in RVE as proposed by Bell and McCaffrey (2002) and extended by McCaffrey et al. (2001) and by Tipton (2015). We will use the Satterthwaite degrees of freedom (Satterthwaite, 1946) for tests as proposed by Bell and McCaffrey (2002) and extended by Tipton (2015). We will use the guidelines provided in Tanner-Smith and Tipton (2014) to evaluate if there are enough studies for this method to consistently estimate the standard errors.

If there is not a sufficient number of studies to use RVE we will conduct a data synthesis where we use a synthetic effect size (the average) to avoid dependence between effect sizes.

Subgroup analysis and investigation of heterogeneity

We will investigate the following factors with the aim of explaining potential observed heterogeneity:

Type of programme, that is, whether it is a universal, indicated or selective intervention; and the three different age-appropriate programmes: ‘Fun Friends’ (4–7 year olds), ‘Friends for Life’ (8–11 year olds) and ‘My Friends Youth’ (12–16 year olds), or the corresponding before 2005 versions ‘FRIENDS for Children’ (7–11 year-olds), and ‘FRIENDS for Youth’ (12–16 year-olds). Other study-level summaries of participant characteristics (e.g., studies considering a specific gender or studies where separate effects for girls/boys are available) and SES indicator (e.g., studies considering a specific SES indicator or studies where separate effects for low/high socioeconomic status are available). In addition, we will investigate other programme characteristics such as type of provider (lay/teacher or mental health provider), country of implementation (Australia/other countries) and implementation issues in relation to the booster sessions and parent sessions (implemented, partly implemented or not at all).

If the number of included studies is sufficient and given there is variation in the covariates (age, gender, SES and programme characteristics), we will perform moderator analyses (multiple meta-regression using the mixed model) to explore how observed variables are related to heterogeneity.

If there are a sufficient number of studies, we will apply the RVE approach and use approximately inverse variance weights calculated using a method proposed by Hedges et al. (2010). This technique calculates standard errors using an empirical estimate of the variance: it does not require any assumptions regarding the distribution of the effect size estimates. The assumptions that are required to meet the regularity conditions are minimal and generally met in practice. This more robust technique is beneficial because it takes into account the possible correlation between effect sizes separated by the covariates within the same study (e.g., age or gender separated effects) and allows all the effect size estimates to be included in meta-regression. We will calculate weights using estimates of τ ², setting ρ = 0.80 and conduct sensitivity tests using a variety of ρ values; to assess if the general results are robust to the choice of ρ. We will use the small sample adjustment to the residuals used in RVE and the Satterthwaite degrees of freedom (Satterthwaite, 1946) for tests (Tipton, 2015). The results in Tipton (2015) suggests that the degrees of freedom depend on not only the number of studies but also on the type of covariates included in the meta-regression. The degrees of freedom can be small, even when the number of studies is large if a covariate is highly unbalanced or a covariate with very high leverage is included, The degrees of freedom will vary from coefficient to coefficient. The corrections to the degrees of freedom enable us to assess when the RVE method performs well. As suggested by Tanner-Smith and Tipton (2014) and Tipton (2015) if the degrees of freedom are smaller than four, the RVE results should not be trusted.

We will report 95% confidence intervals for regression parameters. We will estimate the correlations between the covariates and consider the possibility of confounding. Conclusions from meta-regression analysis will be cautiously drawn and will not solely be based on significance tests. The magnitude of the coefficients and width of the confidence intervals will be taken into account as well. Otherwise, single factor subgroup analysis will be performed. The assessment of any difference between subgroups will be based on 95% confidence intervals. Interpretation of relationships will be cautious, as they are based on subdivision of studies and indirect comparisons.

In general, the strength of inference regarding differences in treatment effects amongst subgroups is controversial. However, making inferences about different effect sizes amongst subgroups on the basis of between-study differences entails a higher risk compared to inferences made on the basis of within study differences (see Schandelmaier et al., 2020). We will therefore use within study differences where possible.

We will also consider the degree of consistence of differences, as making inferences about different effect sizes amongst subgroups entails a higher risk when the difference is not consistent within the studies (Schandelmaier et al., 2020).

Sensitivity analysis

Sensitivity analysis will be carried out by restricting the meta-analysis to a subset of all studies included in the original meta-analysis and will be used to evaluate whether the pooled effect sizes are robust across components of risk of bias. We will consider sensitivity analysis for each domain of the risk of bias checklists and restrict the analysis to studies with a low risk of bias.

Sensitivity analyses with regard to research design and statistical analysis strategies in the primary studies will be an important element of the analysis to ensure that different methods produce consistent results.

Treatment of qualitative research

We do not plan to include qualitative research.

Summary of findings and assessment of the certainty of the evidence

The GRADE (Grades of Recommendation, Assessment, Development, and Evaluation) system will be used to assess the certainty of the body of evidence as it relates to the studies that contribute data to the meta-analyses for the prespecified outcomes (Guyatt et al., 2008). The system classifies certainty of evidence as high, moderate, low, or very low (Guyatt et al., 2013). Five of the eight criteria proposed in the GRADE method have the potential to decrease one's confidence in the correctness of the effect estimates: risk of bias, inconsistency of results across studies, indirectness of evidence, imprecision, and publication bias. Three further criteria are proposed that have the potential to increase this confidence: a large magnitude of effect with no plausible confounders, a dose–response gradient, and a conclusion that all plausible residual confounding would further support inferences regarding treatment effect. GRADE proposes these three criteria should be considered particularly in observational studies. (Guyatt et al., 2011). We will justify all decisions to downgrade or upgrade the certainty of outcomes, and make comments to aid readers' understanding of the review where necessary.

The outcomes will be graded as follows.

High certainty: further research is very unlikely to change our confidence in the effect estimate.

We will use the GRADEpro GDT software (available at https://gdt.gradepro.org/app/) to produce a summary of findings table, presenting the overall quality of the body of evidence according to GRADE criteria for the mental health outcomes.