Abstract
For decades, researchers and providers of adult-directed early behavioral intervention services have made conceptual arguments linking the receipt of such services to improvements in family and family member quality of life. To date, there has not been a review of the literature to empirically evaluate this claim. Therefore, we conducted a systematic review and identified 15 studies contributing 40 effect sizes to evaluate the relationship between the receipt of early behavioral intervention and family quality of life. Our analyses did not detect that there was a robust and meaningful difference in the quality of life of families with a child that did and did not receive early behavioral intervention. Considerations to inform future research are presented; and implications are discussed with respect to parents, pediatricians, providers, and those involved in the public dissemination of evidence-based treatments for individuals with autism.
Quality of Life
Quality of life is broadly conceptualized as an individual’s “sense of fit between themselves and their environment” (Summers et al., 2005, p. 49). In more general terms, an individual’s quality of life is reflective of “what it takes for them to have a good life” (Poston et al., 2003). As discussed by the World Health Organization (2012), there is not a universal consensus on a singular definition or set of indicators that encompass all aspects of an individual’s quality of life; nor is there agreement on the constructs that are most representative of one’s quality of life. Rather, nuanced examinations of quality of life are commonly population-, field-, or discipline-specific.
Across the fields that study autism, efforts have arisen in recent decades to understand the quality of life of families and family members who have a child with autism. Initially, researchers utilized rating scales that were developed for more general populations of families (e.g., Hospital Anxiety and Depression Scale; Hastings et al., 2005) and that focused solely on mental health outcomes such as stress and anxiety. More recently, researchers have identified unique constructs contributing to the quality of life of families with a child with autism (e.g., child’s involvement in social experiences), and in turn, developed measures of quality of life that include these constructs (e.g., Autism Family Experience Questionnaire; Leadbitter et al., 2018). Researchers have also reported on the potential benefits of using qualitative and mixed methods approaches to understand the quality of life of families of a child with autism, as these approaches may provide further insight into constructs that are not captured by rating scales developed for more general populations of families (Kuhlthau et al., 2014). A prominent and consistent finding within this corpus of research is that families of a child with autism have disproportionately worse quality of life outcomes than families that do not have a child with autism. This is reflected across numerous indicators of quality of life (see Beach Center on Disability, 2003), including divorce rates (Hartley et al., 2010; Karst & Van Hecke, 2012), mental health challenges (Myers et al., 2009), health care costs (Sharpe & Baker, 2007), and community involvement (Iadarola et al., 2019). Recent literature reviews on programs designed to improve the quality of life of family members with a child with autism, highlight that mindfulness and cognitive-behavioral therapy demonstrate some promising findings, but that future studies are needed to evaluate these interventions within the context of rigorous and replicated experimental designs (Catalano et al., 2018; Da Paz & Wallander, 2017; Lichtle et al., 2020). As families wait for these interventions to be further refined and evaluated, an understanding about the effects of presently available services on quality of life may help to guide families’ decisions and expectations surrounding the selection and receipt of commonly utilized autism services.
Early Behavioral Intervention
It is here that we arrive at the impetus for our study. One particular service that is commonly recommended by pediatricians and education specialists for families with a child with autism is early behavioral intervention (EBI). The services to which we are referring are those that utilize instructional practices based on operant-learning theory and derived from the discipline of Applied Behavior Analysis. In addition, these services typically rely heavily on adult-led instruction (e.g., discrete trial instruction, verbal behavior approach) rather than child-initiated opportunities for instruction (e.g., naturalistic developmental behavioral interventions; Schreibman et al., 2015). Decades of research on these services have consistently demonstrated that they contribute to improvements in young children’s development across the domains of cognition, communication, social skills, and adaptive behavior (Reichow et al., 2012). Furthermore, EBI services are recognized by professional organizations (e.g., American Association on Intellectual and Developmental Disabilities) and federal agencies (e.g., Centers for Disease Control and Prevention, National Institutes of Mental Health) as representing “best practices for individuals with autism and intellectual disability” (Kennedy Krieger Institute, 2021). In recent years, the number of providers of EBI services has increased dramatically across the United States, due in large part to expanding insurance mandates requiring coverage of these services (McBain et al., 2020). As geographic and financial barriers to accessing these services are lessened, more families can obtain EBI services for their children.
Alongside the child development benefits of EBI, prominent researchers and EBI providers have suggested for decades that EBI may offer residual benefits in the way of improving family and family member quality of life. Volkmar and colleagues (1999) reasoned that since “behavioral interventions can significantly facilitate acquisition of language, social, and other skills” then such interventions are likely to be “helpful in reducing levels of parental stress” (p. 47). In considering family-systems dynamics, Cebula (2012) perceived that improvements in a child’s IQ and communication that are linked to EBI, “would have a positive impact on sibling psychological adjustment” as child gains would “ease interactions and relationships within the family” (p. 848). Similar thinking is reported by community-based providers, as “improved family quality of life” is thought to be a “long-term outcome” of behavioral interventions targeting children’s pro-social behavior (Shepley et al., 2021, p. 111). In a recent Cochrane review of EBI and its effects on children’s developmental outcomes, Reichow and colleagues (2018) stressed that there is a need to empirically validate the conceptual arguments linking EBI services and a family’s quality of life. Given the increasing number of families that are accessing EBI services and the potential for these services to produce widespread improvements in family quality of life alongside advances in children’s development, we think that a synthesis and evaluation of the literature on this topic is warranted. Research questions guiding this review are the following:
Method
All search protocols, coding protocols, study data, syntax for rerunning analyses, and referenced supplemental materials are available for public use and review at https://osf.io/bjva9/. It should be noted that this study is not registered with a review repository.
Search
Inclusion and Exclusion Criteria
Study inclusion criteria were as follows: (a) a delineated sample of children received at least 8 weeks of EBI services targeting multiple developmental domains; (b) EBI services consisted of adult-led instruction utilizing operant learning theory (e.g., Early Intensive Behavioral Intervention, discrete trial instruction); (c) at least one quantitative outcome reported on quality of life for which data were collected during or after the provision of EBI services; (d) study authors specifically noted that a reported measure was meant to capture “quality of life” or related constructs (e.g., physical well-being, psychological health, social relationships; community engagement; World Health Organization, 2012), (e) quality of life measures were collected on a family member of the child receiving EBI services; (f) information was available on the technical adequacy of the employed quality of life measure (i.e., psychometric properties), and (g) study used an experimental or quasi-experimental control group design. The criterion that children receive at least 8 weeks of EBI was guided by published evaluations of community-based EBI services which used an 8 week minimum to inform their analytical samples (Dixon et al., 2017; Leaf et al., 2011). Although some researchers have identified evidence of a dose-response relationship between the receipt of EBI and child outcomes (e.g., Virués-Ortega, 2010), such a relationship is not consistently identified across unaffiliated research teams (e.g., Reichow & Wolery, 2009). Furthermore, given a lack of consistent research evidence to guide determinations about a minimum dosage of services necessary to yield changes in child outcomes, we perceive the guidance offered by evaluations of community-based EBI services as acceptable for establishing a minimum. The criterion that EBI services target multiple developmental domains in children was meant to ensure that a comprehensive treatment model of EBI was provided rather than a focused intervention (Hyman et al., 2020; Wong et al., 2015), as it is the comprehensive models of services have been linked to improvements in family quality of life (Eikeseth, 2017).
Studies were excluded if (a) outcomes solely focused on children’s behavioral or developmental gains, (b) targeted EBI services were provided that addressed a singular skill or goal (e.g., focused intervention designed to decrease a severe behavior and increase a socially appropriate replacement behavior), (c) quality of life instrument lacked any psychometric evidence on the validity or reliability of the measure (e.g., researcher-created Likert-type rating scale), (d) services included elements of operant learning theory but were primarily rooted in theories of change other than behaviorism (e.g., sensory integration therapy in which children accessed rewards for appropriate behavior but regulation of senses was thought to be the mechanism of change for improvements rather than the principle of reinforcement), or (e) outcomes focused on treatment acceptability or consumer satisfaction. We did not include studies in which EBI targeted a singular skill because our focus was on EBI models addressing comprehensive changes in child development as discussed in prior reviews and meta-analyses (i.e., comprehensive treatment models Hyman et al., 2020; Wong et al., 2015). Studies employing treatments solely aligned with Naturalistic Developmental Behavioral Interventions (e.g., pivotal response training; Schreibman et al., 2015) were not included in our review, as this is in keeping with current practices observed across contemporary reviews of intervention studies on behaviorally-based interventions (see Crank et al., 2021). Furthermore, the claims surrounding EBI and its potential impact on family quality of life are largely grounded in interventions derived from operant-learning theory (Eikeseth, 2017), for which researchers of Naturalistic Developmental Behavioral Interventions have made concerted efforts to differentiate between interventions grounded in operant-learning theory and those informed by developmental learning perspectives (Schreibman et al., 2015). Also in keeping with current review practices for behaviorally-based interventions, studies employing a combination of adult- and child-led instruction aligned with operant learning theory were determined to meet our inclusion criteria (see Sandbank et al., 2020). This included studies that reported using a combination of adult-led discrete trial instruction and child-led natural environment teaching. We did not have any restrictions on the setting in which EBI was provided, the language in which a study was published, nor the type of publication. It should also be noted that we did not have an inclusion criterion specific to a child’s age or their having a diagnosis of autism; however, as a preview of our results, all children in our identified studies were reported to be younger than 8.5 years old and have autism or an autism spectrum disorder.
Procedures
The search methods utilized in this review were informed by an unpublished literature review conducted by the first author in partial fulfillment for their master’s degree—the unpublished review shared the same purpose as the study discussed in this article. The first and second authors independently completed the same types and sequence of search procedures to identify studies for inclusion. Consensus agreement was established between the authors to resolve any disagreements regarding study inclusion after the search processes were complete. All search procedures were completed between December of 2020 and March of 2021. First, an electronic database search of Medline, APA PsycINFO, and ERIC was conducted using the Boolean phrase: (“applied behavior analysis” or “applied behaviour analysis” or “ABA therapy” or “early intensive behavioral intervention” or “EIBI” or “intensive home-based intervention”) AND (“parent*” or “caregiver*” or “family” or “sibling” or “mother” or “father” or “guardian”) AND ( “quality of life” or “happiness” or “home life” or “stress*” or “experience*” or “self-efficacy” or “parent-efficacy” or “parenting-efficacy”). A total of 183 results, excluding duplicates, were returned. The title and abstract of each result were screened by the authors for potential inclusion. From the electronic search, 29 full-text articles were reviewed. Second, a backward ancestral search was conducted by reviewing the references of each included study. Third, a forward ancestral search was conducted by locating each included study in Google Scholar and selecting a study’s corresponding “cited-by” link to review all studies that referenced the included study. Fourth, we examined all studies included in prior meta-analyses of Early Intensive Behavioral Intervention (i.e., Eldevik et al., 2009; Makrygianni & Reed, 2010; Reichow & Wolery, 2009; Reichow et al., 2018; Spreckley & Boyd, 2009; Virués-Ortega, 2010), as well as all studies coded as providing Behavioral intervention within the recent Project AIM meta-analysis (Sandbank et al., 2020). Fifth, authors reviewed the publication history for individuals listed as an author on at least two included studies. Sixth, a hand search of Journal of Autism and Developmental Disorders was conducted. Finally, a backward and forward ancestral search was conducted for any article identified after the initial ancestral searches. A total of 15 studies were identified for inclusion in the review (refer to supplemental materials for PRISMA search procedure and eligibility flowchart). One manuscript reported on the results from two separate studies, which utilized different samples of individuals in each study (Cebula, 2012); therefore, we refer to each of these studies separately throughout our review (i.e., Cebula, 2012).
Coding
Study-Level Variables
For each identified study, a series of variables were independently coded by the first and second authors. Consensus agreement was established between the authors to resolve any disagreements after all studies were coded. These variables pertained to child demographics, family characteristics, setting, information about the EBI services received, and measures used in a study. Child demographic variables included the (a) number of children in a study, (b) percentage of children with an autism spectrum disorder, (c) mean age, (d) percentage of female children, (e) percentage of children identified as White (non-Hispanic), (f) mean scores from developmental assessments, and (g) the amount of EBI services received by a child prior to the start of the study (continuous variables coded as number of weeks, hours per week, and total number of hours). Family characteristics included the (a) percentage of dual household families; (b) percentage of families for which a parent did not graduate high school, graduated high school, and had a 4-year degree; (c) percentage of families for which the mother was employed full-time; (d) percentage of families for whom English was not their native language; (e) mean yearly income; (f) percentage of families living in rural communities; and (g) if information was reported on families’ socio-economic status (binary variable coded as 1 = yes or 0 = no). Setting variables included the country in which EBI services were provided (free-response) and the location in which services were provided (categorical variable supporting codes of home, community, research site, or multiple locations). Variables pertaining to the EBI services children received included the (a) implementor of the EBI services (categorical variable supporting codes of parent, therapist, multiple implementors, or varied across children within the study), (b) the average dosage of EBI services provided (continuous variables coded as number of weeks, hours per week, and total number of hours), and (c) if any naturalistic or developmentally-supported intervention components (e.g., Natural Environment Teaching) were included with the EBI services (binary variable coded as 1 = yes or 0 = no). All quality of life measures reported in a study were coded (free response) along with the person completing the measure. Codes for the person completing a measure were initially coded using predefined categories; however, ad hoc codes were developed by the study authors to more accurately reflect all individuals that completed a measure (e.g., mother, father, sibling). The complete coding protocol detailing all variables, codes, and definitions is available as supplemental materials.
Outcome-Level Variables
Within each study, multiple outcomes related to quality of life may have been reported. Multiple outcomes were reported when a study employed multiple measures associated with quality of life or a study reported outcomes across multiple family members (e.g., mothers and siblings competed the same quality of life measure). For each outcome pertaining to quality of life reported within a study, a set of binary variables (i.e., 0 = no, 1 = yes) was independently coded by the first and second authors. The first variable was if equivalence was established between groups of children that received and did not receive EBI. If a study reported that groups were significantly different (α = 0.05) on any tested variable for a particular outcome, then the outcome was coded as not achieving equivalence between groups of children. The second coded variable was if equivalence was established between the groups of parents or other family members reporting data on quality of life. As with coding group equivalence between groups of children, if a study reported that groups were significantly different (α = 0.05) on any tested variable for a particular outcome, then the outcome was coded as not achieving equivalence between each group of parents. The third and final variable that was coded for each outcome was if a random process was used to determine group assignment (i.e., to determine the group of children that received or did not receive EBI services). Consensus agreements were established between the authors to resolve any disagreements after all outcome-level variables were coded.
Data Analysis
Effect Sizes
Effect sizes were calculated as standardized mean differences between groups of families whose children received and did not receive EBI, using Hedges’s g with small sample corrections. Means and standard deviations were used to estimate all effect sizes, as these statistics were reported within all studies allowing for calculable effect sizes. We were unable to adjust for any pre-intervention differences between groups given that none of the studies reported findings across regression analyses or analyses of covariance. Although it would have been possible to use change scores to calculate effect sizes for some of the included studies, this is generally not recommended based on contemporary guidelines for conducting meta-analyses (see Higgins et al., 2021). Therefore, we calculated effect sizes based on measurement occasions occurring during or after the receipt of EBI.
Analytical Samples
Primary
Effect sizes were synthesized from studies in which EBI was compared across groups that did and did not receive the intervention. Three of our included studies compared the receipt of EBI with additional components (e.g., cognitive behavioral therapy) to the receipt of EBI without additional components (Clancy, 2017; Corti et al., 2018; Strauss et al., 2015); and one study compared the receipt of EBI with the receipt of a Naturalistic Developmental Behavioral Intervention (i.e., pivotal response training Stock et al., 2013). In addition, one study did not report statistics to allow for the calculation of effect sizes (Birnbrauer & Leach, 1993). Although these studies met our inclusion criteria for this review, we were unable to include their findings within our meta-analyzed results. In total, 40 effect sizes from 12 different samples of individuals reported across 10 studies were included in our primary analytical sample for synthesizing findings.
Secondary
In reviewing the measures reported across studies, we observed that 80% (n = 8) of the studies in the primary analytical sample reported at least one measure on the construct of stress. Given this finding, we developed an ad hoc secondary analytical sample to synthesize findings from effect sizes specific to differences in stress between families that did and did not receive EBI. Six of the studies contributed a single effect size pertaining to stress. One study reported outcomes related to stress collected at 12 months and 24 months after the receipt of EBI services (Peters-Scheffer et al., 2013); we retained only the 12-month outcome for our secondary analytical sample as this outcome better aligned with the outcomes from other studies (i.e., no other study reported findings 24 months post-intervention). In addition, one study reported the results of two subsections of a measure but did not report a composite score (Kamio et al., 2015); therefore, we calculated the average values from the subsections to create a single effect size from the study. In total, our secondary analytical sample included eight effect sizes from eight different samples of individuals across eight studies (i.e., each study contributed a single effect size).
Meta-Analysis
Primary Analytical Sample
Prior to combining effect sizes across studies, we first ensured that the direction of all estimates was consistent regarding changes in quality of life. That is, some measures assessed self-efficacy, for which higher scores indicated higher levels of self-efficacy and thus improved quality of life. Other measures assessed stress, for which higher scores indicated higher levels of stress and thus worse quality of life. For all measures for which higher scores indicated worse quality of life, we reversed the relationship by multiplying each effect size value by −1. Second, we examined the distribution of effect size values and identified one estimate that was above the outer fence value of the distribution (i.e., outlier value due to being 3 times the interquartile range); we recoded this estimate to equal the corresponding outer fence value.
Given that many of the included studies reported multiple outcomes for family quality of life, we employed robust variance estimations to control for correlated outcomes within each study (see Supplemental Table S2). Robust variance estimations were applied within a random-effects model using Stata 16.0 and the robumeta package (Hedberg, 2011). Using correlations provided by Cebula (2012), we assumed a correlation of 0.50 for our analyses (sensitivity analyses around this correlation did not result in any meaningful changes in coefficient estimates or standard errors; see supplemental materials). To test the effect of EBI on family and family member quality of life, we constructed a null model containing only the intercept. In this model, the coefficient value of the intercept should be interpreted as the average difference in quality of life for families with children that did and did not receive EBI, when controlling for correlated outcomes within studies.
Bias
We intended to conduct a test for publication bias by creating an indicator variable to delineate non-peer-reviewed studies included in our review; however, we had only one one-non-peer-reviewed study in our primary analytical sample. Therefore, due to a lack of variation in the data, we were unable to conduct this test. We did not conduct any other tests of publication bias with our primary analytical sample given that traditionally utilized methods (e.g., trim-and-fill, Egger’s test) have not been validated for use with robust variance estimation in meta-analysis (Pustejovsky & Rodgers, 2019). We attempted to estimate the impact of bias due to parent inequivalence and non-random assignment by creating indicator variables to include in the previously described model but again were unable due to a lack of variation in the data necessary for use with robust variance estimation (Tipton, 2015). Sufficient variation was present to test for the biasing effect of child inequivalence; this variable was coded as an indicator variable for which 10 of the 40 effect sizes in the primary analytical sample established the equivalence between groups of children. This variable was entered alongside the intercept in the previously described model.
Moderators
We planned to conduct moderator analyses of select variables pertaining to service provision models (e.g., home-based), dosage of EBI (e.g., total number of hours of services), and family characteristics (e.g., dual-family household). Due to minimal variance in our data and a lack of information reported across included studies to allow for reliable coding, we were only able to conduct analyses with three variables. These variables were: (a) child age, (b) dosage of EBI services, and (c) if naturalistic or developmental practices were incorporated into the provision of EBI services. Child age and dosage were coded as continuous variables measured in months and weeks, respectively. It should be noted that testing for dosage as a measure of total hours of EBI, rather than number of weeks of EBI, would be preferred; however, too few studies reported information on total number of hours of EBI to test that particular aspect of dosage. The inclusion of naturalistic or developmental practices (e.g., natural environment teaching) within EBI services was coded as binary (i.e., 0 = Not included, 1 = Yes included). Each variable was entered as a single variable alongside the intercept of the previously described model.
Secondary Analytical Sample
In contrast to analyses conducted with our primary analytical sample, we did not reverse the direction of effect for the effect sizes in our secondary analytical sample. Given that all effect sizes in this sample pertained to differences in stress between groups that did and did not receive EBI, negative values in our results should be interpreted as indicating improved quality of life with respect to stress. In addition, given that each study contributed a single effect size to the secondary analytical sample, we did not employ robust variance estimation, and instead used random-effects models and inverse-variance weights for analyses of the overall summary effect and moderator effects. All analyses were completed using Stata 16.0 and meta command frameworks.
Bias
Bias was assessed through various methods. To examine for small-study effects, we constructed a funnel plot and used Egger’s regression test (Egger et al., 1997). Missing study effects were estimated using a trim-and-fill analysis (Duval & Tweedie, 2000a; Duval & Tweedie, 2000bb). We originally intended to estimate the biasing effect of non-peer-reviewed studies within a meta-regression model; however, limited variation in the secondary analytical sample precluded such analysis. The biasing effect of child inequivalence and parent inequivalence was estimated by creating separate variables to indicate if equivalence was established for each corresponding effect size. These variables were then entered into separate meta-regression analyses. We were unable to assess the biasing effect of non-random assignment due to a lack of variation in our data.
Moderators
Similar to issues with testing moderator variables for our primary analytical sample, we were limited in the number of moderator analyses we were able to conduct with the secondary analytical sample. Sufficient data were present to test the following variables: (a) child age, (b) if parents were involved in the provision of EBI services in the home environment, and (c) if naturalistic or developmental practices were incorporated into the provision of EBI services (refer to Supplemental Table S3). We initially intended to test (a) parent involvement in the provision of EBI services and (b) the provision of EBI services in the home environment, as two separate variables. However, preliminary examinations of bivariate correlations between moderator variables indicated a perfect correlation between these variables; thus, we treated these variables as an interaction term within the moderator analysis. Moderator analyses were conducted by entering each variable into a separate random-effects meta-regression model.
Results
Descriptive Information
Children
Information on the children included across studies is reported in Table 1. Across all studies, 84% of the participating children were male and the average age of a child was 55 months. All children were reported to have autism or an autism spectrum disorder. The average IQ of children within studies ranged from 96 to 37. Regarding the ethnicity of the participating children (see Supplemental Figure S1), studies were conducted across seven different countries, with most children being from the United Kingdom (n = 186, 28.2%) and the fewest being from Australia (n = 14, 2.1%).
Information on Child Participants Across Studies.
Note. Com = communication, - = could not be determined.
Reported as standard scores.
Families
Regarding family characteristics, few studies reported information to allow for coding of the variables included in this review (see Supplemental Table S1). Eight of the 15 studies (53.3%) reported information on dual-caregiver households, one study (6.7%) reported on parent education, four studies (26.7%) reported on mothers’ employment status, and six (40.0%) studies reported on families’ socio-economic status. No information was presented across any study to allow for coding of families’ income, primary language, nor rurality. Due to the variation in the country in which studies were conducted, our coding of the percentage of families for whom English was the primary language was not applicable to seven of the studies (46.7%). Given the lack of reported information on family characteristics across the studies and the limited implications that can be derived from these data, we refer readers to our supplemental materials for further review.
Services
Table 2 provides information on the EBI services provided across studies. Multiple types of implementors were most commonly used across studies (n = 7, 46.7%), followed closely by therapist implementation of services (n = 5, 33.3%). Services were most commonly provided at a location in the community (n = 5, 33.3) or in a child’s home (n = 4, 33.3%). Within seven studies (46.7%), EBI services included a naturalistic or developmental component alongside adult-directed intervention. Durations of EBI services ranged from 17 to 104 weeks, with the total amount of services ranging from 34 to 2,662 hr.
Information on Early Behavioral Intervention Services Provided Across Studies.
Note. Nat/Dev = naturalistic/developmental; — = could not be determined.
Study reported that naturalistic or developmentally informed practices were incorporated into the early behavioral intervention services provided; b individual implementing early behavioral intervention services varied across children such that some children may have received early behavioral intervention services exclusively from a therapist and other children may have received early behavioral intervention services from both a parent and a therapist.
Meta-Analysis
Supplemental Table S2 provides information on the quality of life measures that were used in each study for which we were able to calculate effect sizes. Seven different measures of quality of life were used across studies, with versions of the Parent Stress Index being most common. Measures targeted constructs of quality of life related to stress, anxiety, depression, strain, unkindness, avoidance, self-efficacy, support, involvement, empathy, and self-concept. We did not identify any studies that reported on changes in status variables among families, such as employment or marital status. Measures were completed by parents (i.e., study did not make a differentiation between mothers or fathers) in nine studies, mothers in one study, fathers in one study, and siblings in one study.
Primary Analytical Sample
Results of Model 1 reported in Table 3 provide the overall summary effect. An interpretation of the intercept in this model indicates that the quality of life of families with children that received EBI services was on average 0.181 standardized mean difference units (i.e., Hedges’s g) less than families with children that did not receive EBI, when controlling for correlated outcomes. The precision of this estimate precluded statistical significance (SE = 0.120, p = .164, 95% CI = [−0.450, 0.088]), indicating that we cannot reject the null hypothesis of this significance test that the quality of life of families in each group did not differ. The tau-squared value for this model was 0.000, suggesting that sample-level effect sizes were homogeneous.
Random-Effects Meta-Analysis With Robust Variance Estimations for the Primary Analytical Sample.
Note. Estimates reported as standardized mean difference units using based on Hedges’s g; standard errors in parentheses. ES = effect size; dfs = degrees of freedom.
Model 2 in Table 3 estimated the biasing impact of group equivalence at the child level. Results indicated that effect sizes derived from samples in which equivalence was established between children that received and did not received EBI, were on average 0.172 standardized mean difference units higher than studies in which child equivalence was not established. This estimate did achieve statistical significance (SE = 0.347, p = .641, 95% CI = [−0.719, 1.063]).
Models 3 to 5 in Table 3 provide results of moderator analyses. Despite the homogeneity observed in effect sizes in Model 1 based on the tau-squared statistic, we continued with our moderator analyses per our a priori intentions. None of the coefficient estimates achieved, nor approached, statistical significance. In addition, tau-squared remained at 0.000 across the models. Model information and statistics that are not reported may be obtained within supplemental materials.
Secondary Analytical Sample
Figure 1 displays a forest plot of effect sizes along with estimates and 95% CIs. A total of 437 participants contributed to the overall summary effect, with 223 participants in the treatment group (i.e., family members of a child that received EBI services) and 214 participants in the control group (i.e., family members of a child that did not receive EBI services). The overall summary effect indicated that families with children who received EBI services were, on average, 0.218 standardized mean difference units (i.e., Hedges’s g) more stressed than families with children who did not receive EBI. The estimate did not achieve statistical significance (95% CI = [−0.136, 0.572], p = .230); therefore, we cannot reject the null hypothesis that the groups of families do not differ with respect to stress. The Q statistic and its corresponding p-value (Q = 21.43, p = .003) indicated that the dispersion of effect sizes is greater than would be expected by random error. The I2 statistic provided a quantification of this dispersion and indicated that differences not attributable to random error accounted for 66.04% of the variability in the effect size estimates. Tau-squared estimated that the variance between effect sizes was 0.16.

Forest Plot of Effect Sizes and Overall Summary Effect From Studies in the Secondary Analytical Sample.
Results of Egger’s regression test were non-significant (B = 0.530, SE = 1.505, p = .727) which indicated that the test did not detect the presence of small study effects (refer to Supplemental Figure S2). Corresponding to these results, the trim-and-fill analysis did not impute the effects of additional studies due to estimating that no such studies were missing. The results from Egger’s regression test and the trim-and-fill analysis appear inconsistent with the relatively high heterogeneity in effect sizes (I2 = 66.04). Table 4 provides the results of models that estimated the biasing effects of non-equivalence between groups of children and groups of parents. As an interpretation of the estimate for the variable indicating if parent equivalence was established, families with children that received EBI services were on average 0.669 standardized mean difference units (i.e., Hedges’s g) less stressed than families whose children did not receive EBI. The estimate achieved statistical significance (SE = 0.298, 95% CI = [−1.254, −0.085], p = .025] and accounted for 61.61% of the variability in the residuals (R2). In addition, the dispersion of the effect sizes, as measured by I2, reduced by 26.58 percentage points from the model estimating the overall summary effect. In contrast, the variable indicating if child equivalence was established, did not achieve statistical significance and accounted for 0.00% of the residuals’ variability. Furthermore, the inclusion of the child equivalence variable increased the dispersion of effect sizes by 5.02 percentage points, suggesting that the variable added to the observed heterogeneity in effect sizes rather than contribute to explaining the heterogeneity.
Random-Effects Meta-Regression for Effect Sizes From the Secondary Analytical Sample.
Note. Each variable was entered as a single variable in a separate model with the intercept included; estimates for the intercept are not reported in this table but may be accessed via supplemental materials; coefficient estimates reported as standardized mean difference units based on Hedges’s g; standard errors in parentheses; n = 8 for all models. CI = confidence interval.
Parent implemented or assisted with implementing early behavioral intervention services in the home environment.
There was sufficient heterogeneity in the results of the secondary analytical sample, as captured by previously reported statistics (i.e., Q, I2), to warrant an analysis of moderators to explain the variation in effect size estimates between studies. However, none of the three variables we were able to test achieved, nor approached, statistical significance (see Table 4). Furthermore, only the average age of a child accounted for any residuals’ variability (R2 = 13.60). Therefore, none of the tested moderator variables helped to meaningfully explain the variation in effect size estimates of differences in stress between groups. Model information and statistics that are not reported may be obtained within supplemental materials.
Discussion
In this study, we sought to determine if EBI, which has been shown to lead to improved child gains in a variety of developmental domains, also contributed to improvements in family or family member quality of receive EBI services. Although our analyses did not detect a robust and meaningful difference in the quality of life of families with a child who did and did not receive EBI services, this finding should not be construed as indicating that EBI services have no impact on family or family member quality of life. Rather, imprecision in our point estimates across our models preclude a definitive determination about the effectiveness or ineffectiveness of adult-directed EBI services on quality of life. A possible reason for the imprecision in our estimates and the related heterogeneity in our effect sizes is due to the variety of locations in which our included studies were conducted. The 15 studies in this review were conducted across seven different countries and four different hemispheres. We think it safe to assume that there are differences in the culture, customs, and values of the people across (and within) each of these countries, which in turn, will likely contribute to differences in the constructs that each group of people perceive as influencing their quality of life. Thus, if a single measure is used across groups of individuals with differing perceptions of what constitutes a desirable quality of life, then that measure may be more appropriate for some groups of individuals than others. This could be particularly true for measures that examine a single construct, rather than multiple, due to the appropriateness of those measures being dependent on just one construct. As it pertains to our review, the most commonly used measure across all studies was the Parent Stress Index, with measures the singular construct of stress.
Considerations for Future Research
Despite our inability to identify a clear relationship between EBI services and family quality of life, we are able to provide clear recommendations to inform future research. First, quality of life measures should be aligned with the population of families included in a study. At a minimum, quality of life measures should take into account the unique constructs that contribute to the quality of life of families with a child with autism (e.g., Autism Family Experience Questionnaire; Leadbitter et al., 2018). We think it relevant to note that many of the studies included in this review were conducted prior to the development of such measures. If available, quality of life measures should also be sensitive to shared cultural values of the families included in the study (e.g., technical validation of a measure was completed with families from the same region as the families included in a study). Second, researchers should attempt to establish equivalence between families in treatment and control groups. If equivalence cannot be established due to outside processes that dictate assignment, then at a minimum, there should be statistical adjustments made to account for differences between treatment and control groups when analyzing changes in quality of life. Although some studies in our review used statistical adjustments to control for differences in groups when examining child outcomes, these adjustments did not occur when examining quality of life outcomes. Third, to inform future meta-analyses on this topic, researchers should examine studies using qualitative and mixed methods approaches. Our decision for this review to include only quantitative studies was due to the limited expertise of the research team. Given that newly developed quality of life measures appropriate for families with a child with autism may not yet be widely utilized and included throughout the peer-reviewed literature, the examination of already published literature employing qualitative methods may offer insights that can inform the current provision of EBI services and support families in addressing their needs surrounding quality of life. Finally, researchers should collect and report data on family characteristics, such as employment status, socio-economic status, and number of adults living in the household.
Assistance to Readers in Interpreting Our Findings
Throughout the time that we conducted and wrote this review, many personal accounts from individuals with autism and their families have come to light, in which these individuals described unfavorable and sometimes harmful experiences with adult-directed EBI services (Sandoval-Nortan & Shkedy, 2019). In response, practitioners and researchers associated with the EBI community defended such services (Gorycki et al., 2020). Given that these discussions are presently on-going, we are wary that some readers may attempt to cherry-pick findings from this review to support certain narratives. As such, we need to be clear that our results do not suggest that adult-directed EBI was harmful to family or family member quality of life. Likewise, our results should not be used to support assertions that EBI may offer residual benefits in the way of improving family and family member quality of life. That being said, we did identify some evidence to suggest that EBI may improve a family member’s stress level; however, this was conditional on a study establishing equivalence between parents in each group. In addition, the statistical significance of this finding was marginal (p = .025) and based on effect sizes from only eight studies. Considering the number of significance tests conducted in our entire review, finding a single statistically significant p-value may simply be due to random chance. Given that we conducted 11 statistical significance tests across our review, there was a 9% chance that one of those tests may indicate a statistically significant finding due to random chance. If we attempt to correct for the number of significance tests conducted, a Bonferroni-corrected significance level indicates that to be statistically significant at the 5% level, a p-value would need to be at or below a value of 0.005. We think it critical to illustrate the logic behind these corrections to convey that our single marginally significant finding is far from conclusive evidence of the true effect of EBI on family quality of life. In considering this, we urge readers to take a conservative approach to interpreting our findings.
Implications for Providers, Pediatricians, Families, and Related Individuals
For providers of EBI services, pediatricians who serve young children with autism, and those involved in the public dissemination of EBI information (e.g., Autism Speaks, Centers for Disease Control and Prevention, National Institutes of Health), our findings suggest a need to be mindful in how the potential benefits of EBI services are conveyed, in particular to prospective families. That is, providers should clearly articulate that some children experience benefits from EBI with respect to improvement in developmental areas; however, these gains in child development should not be assumed to translate into improvements in a family’s quality of life. Thus, if families are seeking EBI services to address both their children’s needs and their broader needs as a lookfamily, then providers should discuss alternative options specific to addressing the family’s needs (e.g., training on mindfulness techniques; Lichtle et al., 2020).
Conclusion
Studies have consistently demonstrated that adult-directed EBI contributes to developmental gains in some children with autism. In synthesizing the data from our identified studies, we are unfortunately unable to conclude that gains in child development as a result of EBI, will have a residual impact on family or family member quality of life, be it desired or undesired. For families seeking EBI services in hopes of improving some aspect of their quality of life, EBI should not be utilized as the sole treatment; rather, at this point in time, other services should be recommended to treat aspects of family and family member quality of life for families with a child with autism.
Supplemental Material
sj-docx-1-jei-10.1177_10538151221137806 – Supplemental material for Does Adult-Directed Early Behavioral Intervention Improve Family Quality of Life? A Systematic Review and Meta-Analysis
Supplemental material, sj-docx-1-jei-10.1177_10538151221137806 for Does Adult-Directed Early Behavioral Intervention Improve Family Quality of Life? A Systematic Review and Meta-Analysis by Amanda Duncan and Collin Shepley in Journal of Early Intervention
Supplemental Material
sj-docx-2-jei-10.1177_10538151221137806 – Supplemental material for Does Adult-Directed Early Behavioral Intervention Improve Family Quality of Life? A Systematic Review and Meta-Analysis
Supplemental material, sj-docx-2-jei-10.1177_10538151221137806 for Does Adult-Directed Early Behavioral Intervention Improve Family Quality of Life? A Systematic Review and Meta-Analysis by Amanda Duncan and Collin Shepley in Journal of Early Intervention
Footnotes
Author Contributions
The authors are listed alphabetically. Amanda Leigh Duncan conceived the study and contributed to the design, collection of data, interpretation of results, and review of the article. Collin Shepley contributed to the study design, collection of data, analysis of data, interpretation of results, and writing of the article.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
