Oral language interventions can improve language outcomes in children with neurodevelopmental disorders: A systematic review and meta-analysis

The value of a transdiagnostic approach to language intervention

The CATALISE consortium (Bishop et al., 2016, 2017) highlighted clinical assumptions that children with different neurodevelopmental disorders may require different therapeutic approaches or that children with non-verbal cognitive deficits may not benefit from oral language interventions to the same extent that their cognitively able peers do. However, there is limited evidence directly comparing intervention effects across neurodevelopmental disorders on which to make this judgement. Importantly, the language trajectories for children with neurodevelopmental disorders are complex, and there are small to substantial variations in language acquisition both within and across diagnostic groups. In addition, many studies show that there can be pervasive deficits within different subcomponents of language for these children, necessitating assessment across the subcomponents of oral language (Norbury & Paul, 2015). However, assessing language skills in young children in a reliable and valid way is challenging.

A transdiagnostic method that compares children with different neurodevelopmental disorders enables the investigation of unique versus similar approaches. Several primary studies of language profiles have included direct comparisons of different neurodevelopmental disorders. For instance, one study compared children with Williams syndrome and children with “specific language impairment” and reported distinct patterns of syntactic binding (Ring & Clahsen, 2005). Differences in language profiles have also been reported between children with Fragile X syndrome and Down syndrome; in this case, autism symptom severity was associated with language differences between these two groups (Martin et al., 2013; Price et al., 2007). At the same time, children with autism, Down syndrome, Williams syndrome, Fragile X syndrome or an intellectual disability all display some degree of language deficit (Abbeduto et al., 2016; Rice et al., 2005). Therefore, another reason to focus on children with different neurodevelopmental disorders is that there are considerable overlaps in the severity and pattern of language deficit and/or language strengths (Gibson et al., 2013), shared aetiological risk factors (Valenti et al., 2014) and commonalities in cognitive profiles (Raitano Lee et al., 2016). In addition, there are high rates of comorbidity among these groups of children (Abbeduto et al., 2016; American Psychiatric Association [APA], 2013), and diagnostic categories are not as distinct as once thought (Thapar & Rutter, 2015). Nevertheless, whether similar oral language interventions provide similar levels of benefit for children with different neurodevelopmental disorders, or whether different interventions are needed, remains an unanswered question.

Neurodevelopmental disorders included in the review

In this review, we focus on Developmental Language Disorder and associated differentiating conditions identified by the CATALISE consortium (Bishop et al., 2017). These are biomedical conditions in which language impairment is one of a complex set of symptoms, as in autism or intellectual disabilities. These are distinguished from co-occurring conditions, such as learning disorders and attention deficit hyperactivity disorder (ADHD), in which language difficulties occur at higher than expected rates, but are not always present or characteristic of these conditions.

Theoretical approaches to language intervention

Our starting premise is that the language impairments characteristic of neurodevelopmental disorders arise from a complex interplay of genetic and environmental risk factors and chance events (Mitchell, 2018). These risk factors do not affect language directly; instead, they affect the development of brain structure and function in ways that are non-optimal for learning language. In addition, language acquisition is typically an interactive process in which children play an active role. Thus, because of the nature of many neurodevelopmental disorders, the quantity and quality of language input may be disrupted. Therefore, children with language disorders may require additional language input, more exposures to the same input to achieve the same level of learning relative to peers, and/or input that is structured in such a way that it is easier to learn.

Figure 1 shows a theory of change model for the language interventions. As shown in the figure, interventions fall along a continuum distinguished by the extent to which language instruction is explicit or language learning is implicit (i.e., the learner is not aware of the goals of the intervention; Baron & Arbel, 2022). Implicit, or incidental, approaches are grounded in developmental constructive theories of language acquisition through meaningful interactions with the child. These interventions, such as Hanen or Pediatric Autism Communication Therapy (PACT), seek to coach parents on how to optimise their language input to align with the child's attentional focus and how to identify and interpret child behaviours as potentially communicative; thus, they focus on parent-child interactions.

OPEN IN VIEWER

Figure 1

Theory of change model for the intervention.

Milieu therapy techniques that are implemented during child-led play may also be included here. In this approach, interlocutors again follow the child's lead, map language onto the child's attentional focus, recast child utterances to provide more accurate language models and give the child repeated exposures to new words and language structures. Shared or dialogic book reading may be a more structured form of this approach, but this technique uses the same principles of interactive engagement with the child in a developmentally appropriate task, in which key words or structures are highlighted by the adult as they naturally occur within a meaningful story context. A common feature across the different tasks is that the children are unaware of the learning targets and may even be unaware that they are taking part in an intervention. Instead, the environment is engineered to make the desired targets more salient, but learning is incidental and includes increasing the number of exposures to language targets in naturalistic ways.

The second approach aims to circumvent underlying learning deficits via explicit instruction. This approach is more common in older children, and while targets tend to follow a normative developmental sequence, they may also reflect a “language curriculum” in which targets are more closely aligned to educational priorities. In these approaches, children are told the words, grammatical structures or narrative elements they are expected to learn. Definitions or grammatical rules are explicitly taught and modelled by the adult, followed by multiple opportunities for the child to practice producing and/or understanding these structures, typically with feedback provided in fun, child-friendly activities. At the more extreme end of this continuum of practice, correct attempts at language production may be reinforced with external rewards. Most vocabulary interventions use explicit instruction, and grammatical interventions such as “Shape Coding” (Ebbels, 2007) make use of objects, colours and/or shapes to teach rules of grammar.

These approaches, or the hybrid interventions that combine these approaches, appear in the interventions in every neurodevelopmental disorder we have included. Adaptations to the delivery of these approaches may vary according to the severity of the child's language impairment or co-occurring challenges in attention, behaviour or non-verbal cognitive abilities. Typical adaptations aim to increase attentional focus and time on task; they may include visual prompts/timetables; use of rewards; changes to the frequency, intensity or duration of therapy sessions; and provision of the intervention individually or in small groups.

Factors related to children's characteristics that may impact the intervention's effects

Factors related to components and implementation of language intervention that might influence intervention effects

Types of studies

This review includes randomised controlled trials (RCTs) or quasi-experimental (QE) designs without randomisation. Control groups in the studies consisted of “business-as-usual” (BAU), waiting list, passive and active conditions in a domain that did not involve language and reading activities as these may have an indirect effect on other oral language skills. In addition, interventions comparing two language interventions with different delivery components or different dosage were excluded. The language intervention was required to be additional to treatment, as is usual to determine whether the focused work on language provided added value to children with neurodevelopmental disorders. In addition, we excluded single-subject design studies as the results from these studies may not be comparable with the studies examined in our review due to the type of methodology used to assess the efficacy of language interventions. To be included, studies had to report assessments at baseline and after the completion of the training (i.e., a post-test and/or a follow-up) on language outcome measures. This allowed us to evaluate the following: 1) whether the intervention and control groups had comparable characteristics at the beginning of the intervention, and 2) whether the training was effective after the intervention and/or at follow-up.

Types of participants

Studies eligible for the review assessed samples of children from 2 to 18 years with neurodevelopmental disorders, including Developmental Language Disorder or language difficulties, autism, intellectual disability, Down syndrome, Fragile X syndrome and Williams syndrome. We also included studies in which children were described as having “language difficulties” if these difficulties were sufficiently severe on standardised assessment to warrant a diagnosis and/or the children were reported to be receiving specialist clinical or education services for language. In evaluating the eligibility of the clinical samples, we examined information on the children's clinical diagnosis or the clinical assessment and criteria for diagnosis provided by the author(s). It should be noted that there was great variability in the definition of the clinical samples, with some studies describing children's clinical diagnosis or criteria for a diagnosis and others reporting comprehensive information on the clinical assessment through cognitive, developmental and adaptive behaviour measures. In addition, there were studies of children recruited in regular schools and selected through cut-off scores on language assessments (i.e., children with language problems) and others on children attending special schools or clinics. For this reason, we included studies that recruited from clinical caseloads or specialist education provision and/or used at least the 16th percentile (−1 SD) on receptive/expressive tests, as this is a cut-off commonly used to identify children with language difficulties, and there is no consensus on a quantitative cut-off for language disorder (Bishop et al., 2017). We excluded studies on children with primary speech sound disorders these are related to oral-motor function, articulation and dyspraxia, where the primary intervention target is improving speech intelligibility (Cohen, 2001).

Types of interventions

The included studies had to specifically target oral language skills through different techniques ranging from explicit and structured activities (i.e., explicit instruction on vocabulary, narrative structure or grammatical rules) to implicit and broad activities (i.e., shared book reading, general language stimulation). We also included interventions aimed at supporting and enhancing parents’ and teachers’ interactions with children who had neurodevelopmental disorders to optimise their language input and their responses to children's communicative attempts, to facilitate child language development. Studies were excluded if they failed to provide sufficient information on intervention content to judge the focus on oral language or where language was included as an outcome measure but the intervention itself focused on broader behavioural or developmental targets.

Since this review aimed to assess the efficacy of oral language interventions, we excluded the following:

Interventions that were not primarily language interventions but instead targeted many areas to sustain children's joint attention, engagement, regulation and/or primarily social skills (e.g., Joint Attention, Symbolic Play, Engagement and Regulation [JASPER]).

Types of outcome measures

Electronic searches

The last electronic search was conducted in April 2022. In the searches, the following databases were used: MEDLINE, Embase, ERIC and PsycINFO (all cross-searched in Ovid), CINAHL (EBSCO), the Cochrane Library, the Campbell Library, LILACS (Latin American and Caribbean Health Sciences Literature), SpeechBITE, Epistemonikos, ClinicalTrials.gov, Linguistics and Language Behavior Abstracts (LLBA), Scopus Science Direct, Web of Science and Google Scholar. The search included references without any restrictions on year or language. Retrieval experts from the medical library at the University of Oslo supervised the search. A complete list of research terms used for the present review is reported in Supporting Information 1.

Searching other resources

We scanned the reference lists of previous reviews and meta-analyses on language interventions for the specific diagnostic groups in the present review and conducted a hand search of the tables of contents of the following key journals: Journal of Child Psychology and Psychiatry, Journal of Autism and Developmental Disorders, International Journal of Language and Communication Disorders, and Journal of Intellectual Disability Research. Finally, we searched grey literature, including dissertations, reports and conference proceedings via OpenGrey.eu and PDF searches in Google.

Selection of studies

The meta-analysis was pre-registered (Nordahl-Hansen et al., 2019) and conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Page et al., 2021). The flowchart in Figure 2 reports information regarding the overall literature search, process of study selection and final number of studies included. The search resulted in 8195 records after deduplication. The references were imported into the Rayyan software program to conduct screening by titles and abstracts. First, a random sample of 520 records was independently double-screened by one of the authors and a trained research assistant to assess inter-rater agreement. The inter-rater agreement was assessed using Cohen's K was satisfactory, K = 0.82. Any conflicts and questions related to the eligibility criteria were resolved by discussion with the co-authors before proceeding with the screening. After screening by titles and abstracts, 427 references were available; full texts were then screened to check whether the articles met our inclusion criteria. Another 21 records were identified via citation searching, resulting in a total of 448 records assessed for eligibility. At this stage, two different authors (one being the first author, the rest being distributed among the remaining authors) independently screened a random sample of 60% of the records. The inter-rater agreement with Cohen's K was good (K = 0.83). Questions and conflicts were discussed and resolved among the authors, and doubts about the inclusion or exclusion of the remaining 40% of the papers were discussed for studies that needed further assessment and evaluation.

OPEN IN VIEWER

Figure 2

Flow diagram of the search and inclusion of references.

Data extraction and management

The data set of effect sizes was coded in long form, with different effect sizes referring to the comparison between pre-test and post-test (or follow-up) on each row. (A time lag was added to distinguish between post-test and follow-up scores.) Effect sizes (i.e., SMDs; see the section Measures of treatment effects) were calculated from descriptive statistics reported in tables or text (N, M, SD) for the intervention and control groups at the pre- and subsequent assessments (i.e., post-test, follow-up) whenever possible. Several of the studies in this review evaluated the efficacy of the intervention on more than one language outcome measure. When this was the case, all outcome measures of interest were coded. When studies compared the same control group to different treatment groups, we computed the effect size differences between each eligible treatment group and the same control group. Finally, study findings were sometimes reported in multiple reports, and authors were contacted if there was uncertainty about multiple publications of the original study. In addition, if more than one study was described in one report, the findings for each study were coded separately.

We coded information on participants’ characteristics, including the type of disorder and diagnostic status, mean age (in months), sex (proportion of boys), non-verbal IQ and language level at baseline, when available. This information was coded separately for each group, when possible, and then pooled. As for the components and implementation of the language interventions, information on the type of intervention, the focus of the intervention and individual effect sizes included the language outcome measures and the type of indicator provided. In addition, information on settings, delivery agents, session structure of the intervention (individual vs. group intervention) and dosage—defined as session duration (in minutes), the total number of sessions and the number of weeks of intervention—was coded. As for study quality, the modality of recruitment of participants, study design, the status of the control group, type of language test, country, year and type of publication were coded. The whole sample of studies was used to calculate the inter-coder agreement on coding. To accomplish this, the first and third author coded N, M, and SD data for the intervention and control groups for pre-/post-testing or follow-up. Once each coder calculated the effect sizes based on the coded data, Pearson's correlation was used to test inter-coder agreement. The Pearson's correlation between effect sizes was high (r = 0.87), indicating good agreement. Any disagreements were resolved by discussion and consultation of the original article.

Assessment of risk of bias in included studies

Studies included in the review were evaluated using the Cochrane Collaboration's scale for assessing risk of bias in randomised trials (Higgins et al., 2011). The tool identifies five quality indicators examining selection bias (i.e., sequence generation and allocation concealment), performance bias (i.e., participants and personnel unaware of group assignment), detection bias (i.e., blind outcome assessors), attrition bias (i.e., participants’ withdrawals leading to incomplete outcome data) and reporting bias (i.e., selective outcome reporting). Each indicator was rated as high risk, unclear risk or low risk. Two of the authors independently assessed these indicators for each study, and discrepancies were discussed and solved by consensus among the authors.

Measures of treatment effect

The effect sizes were standardised mean differences (SMDs) with Hedges’ correction for small samples (Borenstein et al., 2011). The effects were calculated as the gain observed in the intervention group (post-test or follow-up minus baseline) corrected for the gain in the control group, standardised by the pooled standard deviation at baseline. Specifically, effect sizes and their variances were calculated using the formulae recommended by Morris (2008; cf. index reported as d _ppc2). For the calculation of the effect size variance, we adjusted the values for a pre-/post-test correlation of ρ = 0.5 as a reasonable estimation of pre-post correlations in training studies. Effect sizes calculated from variables with a negative scoring (i.e., with higher values indicating worse language ability, such as error counts) had their sign inverted. Thus, a positive effect size indicated that the intervention group receiving oral language skills training showed a larger pre-/post-test gain compared with the control group. As for the size of the effect, this was evaluated against baseline benchmarks for effect sizes from studies of preK–12 education interventions evaluating effects on student achievement by Kraft (2020). Based on the distribution of 1942 effect sizes from 747 RCTs evaluating education interventions with standardised test outcomes, Kraft (2020) suggested the following benchmarks: less than 0.05 is small, 0.05 to less than 0.20 is medium and 0.20 or greater is large.

Unit of analysis issues

Multiple outcome variables of interest were frequently reported for the same intervention–control group comparison, and multiple comparisons were sometimes reported in the same study (i.e., because a study included two different intervention groups compared against one control group; see the section on Data Synthesis). Therefore, effect sizes were coded as nested within a group comparison and the latter as nested within a study. This structure of dependencies was dealt with in the data analysis using correlated and hierarchical models, with an assumed constant correlation of ρ = 0.7 among effect sizes clustered within the same study (see the section on Data Synthesis).

Dealing with missing data

When the descriptive statistics (N, M, SD) necessary for calculating the effect sizes for any of the outcome measures of interest were not available from the text or tables, data were extracted from the figures and elaborated on if necessary (i.e., SDs were approximated from plotted error bars representing standard errors or confidence intervals and N, if not directly reported). If needed information was still missing, the authors were contacted. When eligible references presented a potential overlap (i.e., they were published by the same research group and described similar language interventions and participants), the authors were contacted to clarify whether the records could describe the same study. When it was clear that different references reported the same data, only records with more information (i.e., outcome measures or data on moderator variables) were included.

Assessment of heterogeneity

Variability in effect sizes and heterogeneity between studies were quantified using different statistics (Borenstein et al., 2017) applied to the meta-analytic models (see details in the section on statistical modelling). First, we used the Q-statistic to test the null hypothesis that there is no variability in the underlying true effect size (either between or within studies). Second, we reported the I ² index to indicate how much of the observed variance was estimated to reflect differences in the true effect sizes rather than sampling error. Third, we provided the estimated standard deviation of the true effect sizes between studies, τ _study, between comparisons, τ _comparison, and between effects, ω.

Assessment of reporting biases

Assessment of publication bias was complicated by the complex structure of the data and by the predictably large heterogeneity. A complex, multilevel data structure implies that a publication bias may arise for different reasons, including one or more single non-significant outcomes being omitted (or reported but p-hacked) by a study, a group comparison not being reported by a study, or even an entire study not being published. We have no means of investigating all these possibilities. In addition, we did not consider the variety of inferential processes that may affect publication bias within each individual study (i.e., type of analysis, use of covariates, corrected vs. uncorrected multiple comparisons).

We employed the p-curve method on the whole set of effect sizes (Simonsohn et al., 2014), as stated in the protocol. The p-curve represents the plotted distribution of p-values, and it depends on both the distribution of effect sizes and their sample sizes. A right-skewed p-curve (i.e., with a prevalence of small p-values) suggests a true non-zero effect size, whereas a p-curve that is left-skewed with a prevalence of p-values just below p = 0.05 suggests publication bias. Unfortunately, the p-curve method may present the problem of not accounting for the presence of dependency structures between the effect sizes, and it is known to perform poorly when there is between-study heterogeneity (Rodgers & Pustejovsky, 2021). The former (but not the latter) problem can be tackled using methods based on meta-regression. Among them, we chose the precision-effect test and precision-effect estimate with standard errors (PET-PEESE) method, which is known to perform comparatively better than alternative conventional meta-analytic methods to assess publication bias (Stanley, 2017).

The PET-PEESE consists of a two-step meta-regression method in which the standard error (first step) and then the variance of the effect size (second step) is used as the moderator for the effect size (Stanley & Doucouloagos, 2014). The second step is performed for a better estimate only if the first step suggests a non-zero true effect size. The final, bias-free estimated effect size is the intercept of the model. We implemented the PET-PEESE method as an additional moderator analysis on the main meta-analytic model.

A related problem with meta-regression tests of publication bias (or the funnel plot) is that the effect size estimate in SMDs is not independent from its variance (Morris, 2008), which inflates the risk of falsely detecting or overestimating publication bias (Zwetsloot et al., 2017). Therefore, we performed the PET-PEESE method on an alternate data set in which the variances were estimated, setting d = 0 in Morris’ (2008) formula. This underestimates variances for non-zero effects, but at the same time, it presents the benefit of providing variance estimates that are orthogonal to the effect sizes and thus usable in funnel plots or regression tests.

Data synthesis

Subgroup analysis and investigation of heterogeneity

Moderator analysis was conducted via meta-regression. Since studies do not always report values for all moderators of interest, there is a predictable loss of information; because of this, we chose to limit the moderator analysis only to moderators for which there was a subset of at least k = 5 studies with complete information. For categorical moderator variables, we performed moderator analysis only on levels of the moderator that were represented by at least k = 5 studies. It should be noted that the moderator analysis was conducted only for gains observed in the post-test because of the limited number of studies that also presented follow-up observations.

The moderators examined in the meta-analysis were as follows:

Participants’ characteristics

Type of disorder. Information on the type of disorder was coded as language disorder, autism, intellectual disability, Down syndrome, Fragile X syndrome or Williams syndrome according to the clinical diagnosis or the clinical assessment and criteria for a diagnosis provided by the author(s).

Components and implementation of language interventions

Type of intervention. Interventions were coded as “explicit instruction,” “implicit/incidental programmes” and “hybrid.”

Indicators of study quality

Recruitment. Information on the modality of recruitment of participants was coded into “specialised centres,” including children on specialist waiting lists and those involved in training programs/therapy caseloads, “schools,” “special schools” and “local agencies and advertisement.”

Sensitivity analysis

For the main meta-analytic estimates, a sensitivity analysis was conducted to see how much the estimate varied with each individual study. This was obtained by recalculating the estimate and removing one single study at each iteration. A second sensitivity analysis was conducted to determine how much excluding studies with implausibly large effects has affected the main meta-analytic estimates.

Summary of findings and assessment of the certainty of the evidence

See the discussion section for a detailed account of this.

Results of the search

After completion of the full-text screening and preliminary check, we retained 42 publications for the analysis (see section below). A detailed description of the effect sizes and group comparisons derived from these publications are reported in the sections below.

Included studies

Excluded studies

We excluded studies that did not present an intervention targeting language skills, did not report any measures of language skills, did not target children with neurodevelopmental disorders or did not include an adequate control group. In addition, language intervention studies on children younger than 2 years and those using multiple baseline designs were excluded. Some studies were shortlisted but ultimately excluded because they failed to report relevant statistics to calculate an effect size, and it was not possible to obtain data from the authors (Figure 2).

Allocation (selection bias)

Blinding (performance bias and detection bias)

Incomplete outcome data (attrition bias)

Selective reporting (reporting bias)

Meta-analytic estimates

Effect sizes for pre-test/post-test comparisons were reported by 38 studies, with 46 group comparisons and a total of 108 effects. The overall meta-analytic estimate of the mean effect was d = 0.27 [95% confidence interval [CI]: 0.15, 0.38]. In line with benchmarks suggested by Kraft (2020), this can be considered a moderate to large effect. The overall true heterogeneity was moderate, I ² = 45%. The model does not estimate heterogeneity in the true effect size across studies, τ _study = 0.00 (meaning that the 95% PI of the true effect size across studies is estimated as having a null range), while the heterogeneity was considerable across group comparisons, τ _comparison = 0.13, and between effect sizes within the same group comparison, ω = 0.21.

Due to the large number of effect sizes, Figure 4 shows the forest plot of effect sizes aggregated by comparison-within-study (assuming a correlation between effects of 0.70), with information of the study and group comparisons in which they are clustered and group the studies by disorder. A sensitivity analysis based on all effect sizes used for the meta-analysis was obtained by leaving one study out of the analysis at a time and recalculating the overall meta-analytic effect with its 95% CI (see Supporting Information: Figure S1). The range of estimates was between 0.23 and 0.31. This was set to ensure that no single study had a major effect on the overall estimate. A second sensitivity analysis re-included all studies regardless of the plausibility of their effect sizes. The estimated mean effect size was larger and much more uncertain, d = 0.35 [95% CI: 0.20, 0.50]. The overall heterogeneity was larger, I ² = 70%. The leave-one-out range of variation was [0.32, 0.40].

OPEN IN VIEWER

Figure 4

Forest plot of effects for pretest-posttest comparisons aggregated by comparison-within-study. Error bars represent 95% confidence intervals (CIs). Larger dots indicate higher precision (i.e., smaller standard errors). The diamond below represents the overall meta-analytic mean effect (its width represents the 95% CI).

Effect sizes for pre-test/follow-up comparisons were reported by eight studies, with 12 group comparisons and a total of 21 effects. The overall meta-analytic estimate of the mean effect at follow-up was d = 0.18 [95% CI: 0.03, 0.33]. (The follow-up was conducted on average after 6 months, with a range of 3–12 months.) The estimated heterogeneity was small, total I ² = 10%, with no heterogeneity estimated between studies, τ _study = 0.00, and minimal heterogeneity between group comparisons, τ _comparison = 0.06, and effect sizes, ω = 0.06. The forest plot of effect sizes at follow-up aggregated by comparison-within-study (assuming a correlation between effects of 0.70) is reported in Supporting Information: Figure S2. Leave-one-out range of variation was [0.16, 0.23]. Sensitivity analysis conducted without excluding any (implausible) effect size, suggested a meta-analytic estimate of d = 0.23 [95% CI: 0.06, 0.40], total I ² = 28%, with a leave-one-out range of variation of [0.18, 0.29].

It should be noted that, for both main meta-analytic models, heterogeneity was estimated as null at the between-study level. Although the models correctly converged, this might be due to imperfect estimation linked with a complex data structure associated with a not so large number of observations. Forest plots in Figure 4 and Supporting Information: Figure S3, however, suggest that this might also be explained by the data: many studies present very large uncertainty bounds, meaning that their deviation from the overall effect might well be explained by sampling error. Furthermore, some of the most clearly heterogeneous effect sizes are indeed reported within the same study or group comparison (e.g., Boyle et al., 2009; Joffe et al., 2019; van der Schuit et al., 2011).

Assessment of publication bias

Because of the limited number of different studies and effect sizes, the assessment of publication bias via p-curve could be meaningfully conducted only for the pre-test/post-test comparison. Figure 5 shows the p-curve analysis, which suggested no risk of publication bias at the level of the entire set of effect sizes. It included 24 statistically significant effects. The distribution of the significant p-values was significantly right-skewed (p = 0.0001), suggesting no anomalous concentration of p-values around .05. The estimated power, based on the distribution of significant effects, was 43% [21%, 65%]. It should be noted that this analysis was based on p-values recalculated from our z scores (not those reported directly by the studies); it did not consider the structure of dependencies among effects, and it could perform poorly as a result of between-study heterogeneity (see the Methods section).

OPEN IN VIEWER

Figure 5

P-curve (Simonsohn et al., 2014) calculated via online app.

We also examined publication bias via the PET-PEESE meta-regression method. This approach allows us to model the structure of the dependencies among effect sizes and provides a bias-free estimate of the effect size. We used an alternate set of variances estimated from the sample size alone to make them independent from effect sizes, but using the same meta-analytic model presented above. For pre-test/post-test comparisons, the PET meta-regression suggested that standard error was a significant positive moderator of the effect size, B = 1.48 [95% CI: 0.49, 2.47], p = 0.004. In other words, small studies tended to yield bigger effect sizes. The estimated bias-corrected effect size (i.e., the intercept) was no longer significant and even became negative, d = −0.13 [95% CI: −0.45, 0.19], p = 0.439. An excess of bias correction is a known possibility in PET meta-regression (Stanley & Doucouloagos, 2014). However, the PET-PEESE method suggests that there is publication bias and that the bias-corrected effect may be null. The funnel plot with the PET meta-regression is shown in Figure 6. For pre-test/follow-up comparisons, the PET meta-regression suggested that standard error was a statistically significant positive moderator of the effect size, B = 2.57, p = 0.020. Once again, the estimated bias-corrected effect size was no longer statistically significant and it was even negative, d = −0.34 [−0.82, 0.13], p = 0.159. The funnel plot with the PET meta-regression is shown in the Supporting Information: Figure S3.

OPEN IN VIEWER

Figure 6

Funnel plot of all effect sizes (visually clustered by study), with PET meta-regression slope and PET-corrected effect size (dot on the top, with error bar representing its 95% confidence interval).

Analysis of moderators

Because of the limited number of studies, moderator analyses could be conducted only for pre-test/post-test comparisons.

Retrospective consideration of power

Our meta-analysis suggested that the true average (net) effect size for pre-test/post-test comparison is most likely between d = 0.15 and d = 0.38 (considering the 95% CI). The overall point estimate was d = 0.27. The overall heterogeneity was modest. As explained above, we adopted the simulation procedure and code offered by Toffalini et al. (2021) to perform power analysis for a pre-test/post-test-control design. All simulations were run with 5000 iterations, assuming test/re-test stability of r = 0.70 and a critical α = 0.05 for statistical significance. Assuming d = 0.15, a sufficient power of 80% would be reached with a total sample of N = 720 (i.e., n = 360 per group); assuming d = 0.38, the same power would be reached with a total sample of only N = 112 (i.e., n = 56 per group). However, for the most likely true effect size, d = 0.27, 80% power is achieved with N = 220 (i.e., n = 110 per group). Nonetheless, power could be increased even with the same sample size, using repeated measurements for each time point and/or using measures with higher reliability (see Toffalini et al., 2021).

Across the studies/group comparisons included in our meta-analysis, the median sample size was N = 43 (about n = 22 per group). None of the group comparisons would reach the required level of power for d = 0.27, as indicated above (the largest sample size for a single group comparison was 200, including 99 treated individuals and 101 controls). Considering the median sample size of 43 (i.e., n = 22 per group), the median power of the studies that we reviewed could be estimated at 22% for d = 0.27; considering the uncertainty bounds, it could be estimated at 10% for d = 0.15 to 39% for d = 0.38. The exaggeration ratios (i.e., the ratio between the estimated effect size associated with p < 0.05 and the assumed true effect size; Gelman & Carlin, 2014) is 2.02 for the point estimate, and 3.47 to 1.53 for the uncertainty bounds, suggesting that when the median study finds a statistically significant effect, it is poised to substantially overestimate the true treatment effect.

It should be noted that the actual power might even be much lower than indicated above. This is because our assessment of publication bias via the PET-PEESE method suggested that the true effect size may even drop to zero after correcting for publication bias.

Are oral language interventions effective to attenuate language problems in children across different neurodevelopmental disorders?

There are initial indications that oral language interventions can yield moderate effects on language. The majority of the intervention studies eligible for our meta-analytic review targeted children with language disorder and children with autism. Effect sizes of the effectiveness of oral language interventions across these two groups were similar, showing that interventions for these neurodevelopmental conditions had moderate effects in attenuating language problems at post-test. Evidence on follow-up evaluations, available for only eight studies, pointed to a smaller but still positive and statistically significant effects after an average of 6 months from the end of the intervention. However, this conclusion is tempered by evidence of publication bias. Publication bias analysis showed that results at post-test and follow-up were overestimated and that the true effect sizes were no longer significant after correcting for publication bias. Pre-registration and replications of more robust and adequately powered trials—including a wider range of diagnostic conditions and more extensive reporting on potential moderator characteristics—are needed to drive evidence-based practice and policy.

What factors do moderate the response to treatment?

We tested the moderating role related to response to treatment by examining participant characteristics, components and implementation of the language interventions, and indicators of study quality. As for participants’ characteristics, children's diagnosis, diagnostic status, age, sex, non-verbal cognitive ability and severity of language impairment at baseline were not statistically significant. The same occurred for moderators regarding the implementation of the language interventions, such as characteristics related to the intervention content, setting, delivery agent, session structure of the intervention or total number of sessions. However, language outcome measures and aspects of dosage were associated with variation in treatment outcomes. Receptive vocabulary and omnibus receptive tests showed smaller effect sizes compared to expressive vocabulary, grammar, expressive and receptive discourse, and omnibus expressive measures. In addition, longer sessions conducted over a longer period of time were more beneficial than brief sessions and short-term interventions. Finally, indicators of study quality (recruitment, study design, status of control group, type of outcome test, country, and publication year) were not statistically significant moderators. Notably, the meta-analysis had limited power to detect differences in the moderator analysis, largely because most studies to date have limited power to detect expected modest treatment effects.

Our findings align well with the meta-analytic review by Rogde et al. (2019) on the effects from oral language interventions on standardised measures of language and reading comprehension. As for the language outcomes, Rogde et al. (2019) showed a mean effect of d = 0.16. As for earlier reviews in this area, for instance, Law et al. (2004) showed very large effects on vocabulary (d = 0.98) and grammar (d = 0.70), Cirrin and Gillam (2008) demonstrated also large effects on vocabulary (d between 0.5 and 3.5), Cirrin et al. (2010) also showed large effects on language measures but also a lot of variation between studies (effects ranging from d 0 to 1.65). Note that these studies focused on children with primary language difficulties and no other developmental concerns, and were concerned with delivery of intervention by qualified speech and language therapists. Overall, our review in general showed lower effects than previous reviews when different neurodevelopmental disorders are examined and publication bias is taken into account.

There can be several reasons for the discrepancy between our review and previous reviews. One apparent reason is that many of the previous reviews included studies with less well-controlled designs than ours. It has been demonstrated that intervention studies that are less well-controlled typically overestimate effects (Lipsey & Wilson, 1993). Thus, as study quality have been improved over the years towards more RCTs, it is also likely that we will experience a downward trend in effect sizes. Another issue that might explain this discrepancy is that many of the previous reviews code outcomes that are constructed by the researchers and contain elements from the training, together with standardised outcomes. Such research made, bespoke measures generally yield much larger effects than standardised measures (Rogde et al., 2021).

As for the moderator analysis, receptive vocabulary and omnibus receptive tests showed smaller effect sizes compared to expressive vocabulary, expressive language, and omnibus expressive measures, which is not surprising given previous studies of these kinds of language interventions. Typically at pre-test, previous studies find that the different language tests constitute a global oral language construct. However, at post-test, most studies do not realise measurement invariance, mainly because the training has affected the measures in the global language construct unevenly (e.g., see Hagen et al., 2017; Rogde et al., 2016; West, Snowling et al., 2021). Typically there are larger impacts from the training on expressive measures than on receptive measures (Hagen et al., 2017; Melby-Lervåg et al., 2020; Rogde et al., 2016). Thus, these kinds of interventions do not seem to be able to improve the language construct per se, only aspects of it. However, this does not mean that the improvement is without value, expressive language is important and improving this might have positive repercussions, for example in the classroom or social participation. It also opens up the possibility that the improvement or some of the improvement might reflect that children receiving the intervention become less cautious and shy, and are better able to use their existing language in social settings. If this is the case, then the effects might not be caused by improvement of aspects of the language construct, but rather in children's willingness to use the language skills they have. Thus, in the context of active control groups (rather than BAU or passive control groups as here) the expressive effects might be reduced.