Abstract
This Campbell systematic review's objective is to summarize the most dependable evidence on the effect of parental involvement intervention programs for improving the academic performance of elementary school age children. The most dependable evidence is defined as studies that include at least two groups and use random assignment to form a fair comparison between groups.
19 Randomised Controlled Trials met the inclusion criteria for the systematic review. The overall findings were that parent involvement has a positive and significant effect on children's overall academic performance. It is also clear that the effect is large enough to have practical implications for parents, practitioners, and policymakers (d= 0.45).
The overall effect suggested that when parents participated in academic enrichment activities with their children outside of school, the benefits were manifest in improved academic performance in school. This result was striking when one considers that the median length of parent involvement was only 11 weeks. This review has uncovered compelling support for the use of a parent involvement program as a viable supplementary intervention to improve children's academic performance in school, and for the parent involvement component of the No Child Left Behind mandate.
Linked article:
Background
Introduction
The impact of parent involvement in a child's growth and development is generally accepted (Sheldon, 2003). However, educators, parent groups, and policy makers continue to debate the issue of whether or not parent involvement has a beneficial effect on the academic achievement of children (Epstein, 2001). A key element in these debates is how parent involvement is defined (Epstein, Sanders, Simon, Salinas, Jansorn, & Voorhis, 2002). Therefore, it is important to understand how parent involvement is defined before conclusions are drawn on the impact of parent involvement. In addition, it is important to understand what aspects of parent involvement have the greatest impact and whether the impact is consistent across children characteristics such as socioeconomic status, race, and the child's grade level, age, and gender. During the past several decades, there have been numerous primary studies investigating various aspects of parent involvement and the effect it has on children's learning. The No Child Left Behind Act (NCLB) has served to focus attention on the need and importance of parent involvement in their child's education.
This systematic review synthesizes findings from this research. For this review, parent involvement is defined as the active engagement of a parent with their child outside of the school day in an activity which centers on enhancing academic performance. This review is intended to provide evidence to policymakers that can guide their decisions about the level of investment in parent involvement, to educators that can guide the development of parent involvement programs for their school improvement plans, and to researchers in designing studies to rigorously investigate the effectiveness of parent involvement for improving elementary school children's academic performance in schools.
Findings from Primary Studies
The role that parents play in the academic achievement of their children has long been thought to be a centrally important one. However, it was not until the 1960's that the effects of programs designed to foster the role of parent involvement were systematically studied using an experimental design. Evaluation of the Head Start Program in the United States focused nationally on outcomes related to parent involvement (Coleman, Campbell, Hobson, McPartland, Mod, Weinfeld, & York, 1966). The results of this evaluation suggested a substantial relationship between the parent's involvement in their child's education and their child's success in academic domains. Subsequent studies supported the findings from Coleman, et al. (Duff & Adams, 1981; Henderson, 1987; 1988). Even so, quantitative evidence on the effect of parent involvement on student achievement has been mixed. For instance, researchers have reported effect sizes ranging from positive to negative to no significant differences between experimental and control groups (Griffith, 1996; Heller, & Fantuzzo, 1993; Henry, 1974; Keith, Reimers, Ferman, Pottenbaum, & Aubrey, 1986; Ryan, 1964; Searles, Lewis & Morrow, 1982).
Some of the discrepancies across studies can be explained by the nature of the data collection and research design. For example, some investigators studied the relationship between parent involvement and a child's school success using direct observation (Arbuckle & MacKinnon, 1988), surveys or questionnaires, or both (Edwards & Warin, 1999). Other investigators mounted traditional experimental designs to compare student performance across randomly allocated groups (DeBaryshe, 1993; Woods, Barnard, & TeSelle, 1974).
Another potential source of discrepancy in the findings across studies relates to outcomes measured. Specifically, a variety of dependent variables have been reported in studies on parent involvement including reading achievement (Epstein, 1987; 1991; Tizard, Schofield, & Hewison, 1982; Trovato & Bucher, 1980; Walberg, Bole, & Waxman, 1980; Woods, Barnard, & TeSelle, 1974), math achievement (Fantuzzo, Davis, & Ginsburg, 1995; Heller & Fantuzzo, 1993; Morgan & Sorensen, 1999), and perceptual skills training (Garrison, 1977). Furthermore, differences in study findings and the conclusions often do not take into account other important factors that can affect the validity of study findings such as the reliability of scales and tests; controlling for important child background characteristics such as grade, age, socioeconomic status (SES); and controlling for important parent background characteristics such as SES, education and training (Reynolds, Weissberg, & Kasprow, 1992).
In addition, researchers have defined parent involvement differently or so broadly that it is difficult to understand how to measure it consistently. For example, one group of researchers defined parent involvement as parent participation in educational activities at both school and home (Christenson, Rounds, & Gorney, 1992). Epstein (1987) suggested that parent involvement is multi-dimensional and included: (1) parents providing a home environment that supports learning, (2) communication between parents and teachers on classroom performance, (3) parent's active attendance at school activities such as PTA, (4) parent's engagement and monitoring of home learning activities, and (5) parent's participation in school-based decision making such as school committees.
Typically, the study of parent involvement in home-school environments has identified educational activities that ranged from parent participation in parent-teacher organizations and conferences to specific training in activities designed to enhance an academic skill such as reading or math proficiency. The vast majority of the studies on parent involvement have focused on elementary school age children (Keith & Cool, 1992). Most schools in the US and abroad have some component of parent involvement integrated into a child's daily activities. Thus, differentiating the effects of parent involvement from other factors that affect student achievement can be difficult.
Findings from Narrative Reviews
In addition to the many primary studies on the statistical relationship between parent involvement and student achievement, several narrative summaries of parent involvement have been published over the past 20 years. Gordon (1977), for example, reviewed parent involvement programs and proposed a three dimensional model: (1) Parent Impact, (2) School Impact, and (3) Community Impact. Gordon's summary indicated that all three models produced positive effects, however, the summary was made without a quantitative analysis of the degree or nature of the intervention effects for any model. This narrative summary provided promising framework to further examine the effect of parent involvement on their children's academic performance.
In a more comprehensive review than Gordon's (1977), Christenson, Rounds, and Gorney (1992) examined over 160 studies that described variations on parent involvement. These reports focused on family components of parent expectations and attributions, structure for learning, home affective environment, discipline, and parent involvement. Christenson, et al. concluded that while there is a positive correlation between student achievement and parent components, there are background variables underlying parent involvement components and student achievement. That is to say, the influence of other variables such as SES, ethnicity, or gender had not been adequately accounted for while studying the interactive nature of the intervention of parent involvement and student achievement.
In a recent synthesis, Mattingly, Prislin, McKenzie, Rodriguez, and Kayzar (2002) reviewed a total of 41 studies described as ‘evaluations’ of parent involvement programs. While this review did not provide a meta-analytic summary of these studies, the authors concluded that there was little evidence to support the efficacy of parent involvement programs to improve student achievement. Mattingly, et al. indicate that their results do not suggest that parent involvement programs are not effective but that the quality of the evidence is compromised due to methodological weaknesses.
Findings from Meta-Analyses
The first meta-analysis on the topic of parent involvement was conducted by Graue, Weinstein, and Walberg (1983) who quantitatively synthesized 29 empirical studies with various types of study designs conducted from 1970-1980 that focused on parent involvement and student academic success. Graue, et al. assessed a wide variety of parent involvement studies without specifically measuring the effect of a particular type of parent involvement, such as joint book reading or homework. Graue, et al. concluded that there was a significant academic advantage for those children whose parents participated in their child's educational program. However, the researchers did not investigate study design type (e.g., experimental versus quasi-experimental) or intervening variables (e.g., length of parent involvement).
Several meta-analyses have been reported in the area of parent involvement and can generally be categorized into one of two types of studies: correlational meta-analyses that assess the nature of the relationship between parent involvement activity and measured outcomes; or intervention effect meta-analyses that assess the impact of a formal program to change measured outcomes. We recognize that several correlational meta-analyses have contributed much to our understanding of the nature of the contributing variables and conditions. Rosenzweig (2000) assessed the association between parent practices and school success across 34 studies representing seven classes of outcome variables. She concluded that the 20 specific parent practices could account for 23% of the explained variance in children's academic performance. No attempt was made to address the causal impact of parent involvement on student achievement.
Similarly, Fan and Chen (2001) reviewed 25 studies that focused on the relationship between parent involvement and academic performance. They found that the overall strength of relationship was modest r=.25 but non-trivial. Fan and Chen's review focused on a correlational analyses rather than an intervention effect summary.
These correlation meta-analyses have certainly painted a fuller picture of the nature of parent involvement in improving the academic performance of children. However, two recent intervention effect meta-analyses were reported that provide an ongoing summary and discussion of the intervention effects of parent involvement programs (Jeynes, 2003; 2005). In the 2003 review, Jeynes examined the effect of various types of parent involvement on academic outcomes for minority students. Parent involvement was broadly defined as ranging from parent expectations of academic success to parents attending or participating in school functions.
This important meta-analysis significantly advanced the understanding of the effects of minority parent involvement on student achievement from an intervention or programmatic perspective. Jeynes conducted an extensive literature search of some 25 databases that produced 26 potential studies of which 20 had useable data and were included in the metaanalysis. Four different measures of academic achievement were identified for the review including a composite of the three academic measures. Jeynes found substantial positive effects of parent involvement for African Americans (d=0.31), Latinos (d=0.48), and Asians d=0.22). While the author did provide discussion regarding validity issues for the cohort of studies, no specific analysis of the quality of design was conducted as part of the meta-analysis.
In 2005, Jeynes reported a second study of the effects of parent involvement programs in which a total of 41 studies were coded and analyzed to address four research questions focusing on the general question of the degree to which parent involvement can improve educational outcomes for urban children. This study provided a more detailed expansive analysis of parent involvement programs by addressing important methodological issues of design, controls, and transparency of effect size calculations. Further, he provided an analysis of differences associated with the more direct parent involvement programs compared to the voluntary or pre-existing programs. While this review did provide for an assessment of the impact of what he termed “sophisticated controls” a more detailed analysis of the experimental design characteristics would seem warranted.
The Need for a Systematic Review
The weight of evidence for parent involvement as a mechanism for improving academic performance in children has been generally accepted but difficult to assess. Narrative reviews have included bibliographic searches and annotated bibliographies. Most concluded that there was a positive result for children whose parents were involved in their academic learning. However, one may question the validity of these conclusions given that the reviewers must subjectively evaluate numerous study characteristics and outcomes. (Bronfenbrenner,1974; Fantini, 1979; Henniger, 1979). Certainly some summaries such as Mattingly, et al. (2002) have produced conclusions contrary to others in the field (Jeynes, 2002, 2005). In addition many of the primary studies reported in the literature were more qualitative than quantitative. Even among the quantitative studies, the evidence is not clear and consistent as to whether parent involvement has a positive, negative, or null effect on elementary school age children's academic achievement. The reason for the lack of clarity is that researchers often have relied on retrospective surveys, correlational studies of existing conditions, and anecdotal observations of both the quality of parent involvement and student achievement.
We cannot turn to the two meta-analyses on the topic to reconcile the mixed conclusions reported in primary studies, because of their methodological limitations. At best, these meta-analyses suggest that parent involvement has a positive effect on achievement.
The purpose of this review is to apply a set of a priori systematic procedures leading to a meta-analysis of studies with adequate controls to produce the least biased estimate of the effect of parent involvement on student achievement. To control for variation in threats to internal validity of a particular study design, we limit our review to studies that include at least two groups and use random assignment to form a fair comparison between groups. In other words, we limited our review to studies that use a randomized controlled trial (RCT) design.
Objective
This review's objective is to summarize the most dependable evidence on the effect of parental involvement intervention programs for improving the academic performance of elementary school age children. The most dependable evidence is defined as studies that include at least two groups and use random assignment to form a fair comparison between groups.
Method
Information Retrieval
Following the policies and guidelines in the Campbell Collaboration's Information Retrieval Methods Group (C2 IRMG) Policy Brief (2004), we considered the following information retrieval tasks and related activities: Electronic Searches of Databases, Hand Searches of Journals, Contacting the Invisible College, and Contributing to C2-SPECTR.
For the electronic searches, we documented our search strategies as recommended in the C2 IRMG Policy Brief by presenting tables that contain the following information: Database Searched Database Supplier (e.g., Ovid), Time Period Searched (e.g., 1966 to 2003), Search Strategy, and Other notable features (e.g., whether database access required a subscription or whether the search was conducted in another language).
The purpose of this documentation is to enable other researchers to replicate our searches. Prior to searching, we consulted with Anne Wade, the Education Coordinating Group's Information Retrieval Liaison and Alan Gomersall and Lesley Grayson of the UK Center for Evidence Based Policy and Practice. They provided feedback on our target list of databases to search and on our search strategies used with these databases. A number of the databases on this list required special access, especially those outside the US. Anne Wade and Lesley Grayson searched those databases on our behalf.
Electronic Searches of Databases
We began with a search of 27 electronic databases. The purpose was to be as comprehensive as possible and thereby reduce bias in the search for evidence. Our information retrieval experts advised that we include databases most likely to catalog studies on parent involvement. We then modified the list by removing some databases and adding others. For each database, we used the following search keywords to locate potentially relevant studies for the review:
We modified these terms to conform to each database's thesaurus, when a thesaurus was available, and to ensure that the universe of appropriate synonyms had been used during the search. Search term syntax was adapted to the requirements of specific databases. The process of locating relevant studies for review was iterative. Thus, for some database searches the final search strategy contained far fewer terms than those presented above.
Table 1 presents the results of our search of 27 databases supplied by entities in the United States, Canada, China, and the United Kingdom. Our intent was to include databases supplied by entities in other countries such as France and Germany. However, access to these databases (such as FRANCIS and SOLIS) was restricted to fee-paying subscribers. Still, as Table 1 shows, our search was comprehensive enough to retrieve over 800 citations with PsycINFO and ERIC yielding the highest number of citations at 205 and 191, respectively. On average, our search strategy retrieved 30 citations per database.
We searched databases such as ERIC, Chinese ERIC and PsycINFO, and discovered that they catalog a significant amount of grey literature which consists of unpublished reports such as dissertations, organizational reports, and government documents. In addition, a number of databases that we searched, such as Social Science Research Network, Index to Theses, and Digital Dissertations only catalogued grey literature. Our ability to expand our searches to databases that catalog grey literature, especially those in Europe, was limited because most of these databases required a subscription.
Hand Searches of Journals
Parental involvement is an interdisciplinary topic and studies on it are published in a wide variety of journals. Originally, we planned to conduct hand searches of specific journals based on the number of randomized controlled trials published in a particular journal as identified through our searches. However, the majority of the randomized controlled trials were identified in the grey literature (dissertations specifically) for which hand searching is not possible. The randomized control trials we identified in academic journals were widely distributed across journals such that a hand search was neither an efficient nor a cost-effective use of human or economic resources.
Contacting the Invisible College
Previous research in the social sciences (Glass, 1981) and more recent research in the health sciences (Hopewell, Clarke, Lusher, Lefebvre, & Westby, 2002) have shown that including grey literature in a meta-analysis can reduce bias. How much bias is reduced will vary according to the amount and quality of the grey literature excluded from the review, and the review topic (Egger, Juni, Bartlett, Holenstein, & Sterne, 2003). For instance, unpublished studies reside in the file cabinets of researchers because the results of the study were not statistically significant. These, are particularly difficult to obtain without personal communication. To identify such studies, an email was sent to over 1,200 scholars, researchers, policymakers, and practitioners worldwide informing them of our review and requesting a referral to a potentially relevant study, a person who might know of such studies, or an organization that sponsors, conducts, or has an interest in parent involvement studies.
Seventy members of the invisible college responded to the email campaign. Of these, 17% (12/70) referred us to a study; 10% (7/70) referred us to an organization; and 70% (49/70) referred us to a researcher. Ten percent expressed interest in the review and requested a copy of the published version.
When respondents emailed us the full-text for a study, we included the study in the initial screening pool and reviewed the reference list for additional leads. When respondents emailed us a citation, we searched the appropriate database to retrieve the abstract and if it was relevant retrieved the full text for further screening. To follow up on referrals to organizations and researchers, we sent the same email and followed the same process as when studies were referred to us.
None of the referrals led to our identifying a randomized controlled trial. However, we did collect a list of researchers, policymakers, and practitioners that expressed an interest and requested a copy of the final review.
Contributing to C2-SPECTR
All citations of studies identified during the information retrieval phase of this review and identified as a RCT or systematic review (that could have references to RCTs) were denoted as such with an asterisk in the tables of included and excluded studies. Studies not relevant to this review might be relevant to other C2 reviews and therefore are relevant for inclusion in the Campbell Collaboration's web-based Sociological, Psychological, Educational, and Criminological Trials Register (C2-SPECTR). As recommended in the IRMG Policy Brief, these tables will be forwarded to the person in the Campbell Collaboration currently responsible for cataloging studies in C2-SPECTR.
Definition of Parent Involvement
The area of parent involvement has been studied under at least three different rubrics: involvement, programs, and interventions. Further, there is a distinction between the ‘voluntary’ or ‘support’ aspects of parent involvement such as school visits, and the more direct activities such as the parent reading with the child. This review uses a focused definition of parent involvement that reflects the implementation of a program in which the parent has a direct interaction with the child in either the delivery or monitoring of the program of intervention. Further, since both Mattingly, et al. (2002) and Jeynes (2005) use the term ‘program’ rather than intervention to describe the experimental condition we will use the term program to refer to what may be typically thought of as the ‘parent involvement intervention or approach’.
Inclusion Criteria for Studies
Included studies reported the following characteristics of the intervention program: 1) Parent involvement with their child in academic support activities outside of school (e.g., reading or completing supplemental math problems with the child), and 2) Parent involvement (as defined in “1.”) for a minimum of 20 days.
Included studies reported the following outcomes on children's academic performance in: reading, mathematics, spelling, writing, language arts, or science.
All included studies assigned participants to an experimental group and control (or comparison) group to create a fair comparison. A control group was defined as a nonintervention condition whereas a comparison group was defined as an alternative intervention condition.
This review was limited to RCTs in order to provide the least biased estimate of the effect of parent involvement on student achievement and to control for variation in threats to internal validity. Glazerman, Levy, & Meyers (2003) have demonstrated that compared to RCTs quasi-experiments tend to produce biased results and found it difficult to identify an aggregation strategy that removed that bias. In all that follows, a randomized trial is defined as a trial in which the participants were assigned prospectively to one or more alternative forms of intervention using a process of random allocation such as a random number generation or coin flip.
Once all potentially topic-relevant and research-design appropriate studies were retrieved, two reviewers evaluated each study independently for inclusion in the review. If there was a disagreement on the inclusion of a study, the source of the disagreement was identified and reconciled to a final decision. If a final decision still could not be reached, the full-text version of the study was submitted to the third reviewer for a decision. Reviewers were not blinded at any level of review to the name(s) of the author(s), institution(s) or publication source.
Retrieved studies were initially screened for topic relevance and research design quality based on the title and abstract. If the study's topic relevance or research design appropriateness could not be ascertained from the title or abstract the full text version of the study was retrieved. Differences of screening decisions were resolved by discussion of the two reviewers. If a final decision still could not be reached, the full-text version of the study was submitted to the third reviewer for a decision.
Coding Categories for Included Studies
Two reviewers independently coded each study for the characteristics that follow.
Design Characteristics
We evaluated the following characteristics of included randomized controlled trials (RCTs) in the review based on the following criteria if reported: Method of random assignment Attrition (overall and differential)
Two reviewers evaluated each study for attrition rates. Attrition rates were calculated based on a reduction of the number of participants from the pre-intervention to the post intervention measures. Studies were coded for the percent of participant attrition for both intervention and control groups.
Participant Characteristics
Each study reported data on a minimum of five participants in both experimental and control (or comparison) groups. The minimum number of group participants was selected in order to maintain the interpretability of the mean and standard deviation that were used in effect size and standard error calculations. Specific child participant characteristics for both the experimental and control (or comparison) groups were coded to include the following: Age Grade Gender Number of Participants SES Ethnicity
Intervention Characteristics
All intervention programs were implemented for a minimum of 20 school days (or four weeks) as this was considered to be a potentially minimally meaningful length of time that a school based program might be implemented and reflect substantive results. No studies were excluded due to length of the intervention program. All intervention programs involved planned and consistent direct contact between the parent(s) and child in an educational activity outside the school-day setting. Studies in which parents were engaged in a classroom activity as an aide or support personnel were excluded from the review. In addition, intervention programs that measured parent involvement for participation in school-related activities such as parent teacher organization meeting, school program attendance, or home visits were also excluded from the review.
We chose a focused inclusion and exclusion criteria for the intervention program for several conceptual reasons. First, it is the case that many parents work during the school day. A comparison of parent involvement activities occurring during the school day could potentially confound outcome interpretation of studies in which parents were not able to participate during schools hours. Secondly, we were interested in the direct impact of parent involvement on achievement and we viewed supportive activities as an indirect intervention program. We recognize that these ‘support’ activities are potentially important and viable strategies for improving children's academic performance and warrant further consideration.
The following approaches to parent involvement were identified in this review.
Outcome Characteristics
Outcome measures for each intervention program contained a quantified assessment of educational performance using norm reference tests, criterion referenced tests, or rating scales of academic performance with appropriate evaluation of each measure's psychometric properties.
Timing of Outcome Measurement
Outcomes were measured at various points in time after the intervention was completed. Only those outcomes reported immediately post-intervention were included in the calculation and synthesis of the intervention's effect.
Data Extraction
Two reviewers extracted data from articles independently using a coding form to classify study methods, participant characteristics, intervention program characteristics, and outcomes. Uncertainty and disagreement between the two coders were resolved through discussion and consultation. If agreement could not be reached, a third reviewer was an arbiter. Consensus was reached for all coded studies in their final form reached consensus. If further information (e.g., missing data for effect size computations) was required in order to conduct appropriate analyses of outcomes, the first author of the study was contacted. When the requested data could not be obtained from the first author, the study was excluded from the analysis.
Effect Size Computations
Effect sizes from studies reporting continuous outcomes were calculated under the assumption of independent groups. Appropriate formulas, as documented in CMA 2.0, (citation) were used to calculate a standardized mean difference with a small sample correction (Hedges g) from the following types of data: Post-test Means, Standard Deviations, Sample Size Difference in Post-test Means, Common Standard Deviations, Sample Size t-statistic, Group Sample Size Difference in Post-test Means, Common Variance, Group Sample Size
The formulas were implemented using CMA 2.0 (citation).
Synthesis of Effect Sizes
All studies reported outcomes on a continuous scale from posttest measures. Using the standardized mean difference (e.g., Hedges's g), an effect size from each study was weighted by the inverse variance and averaged to arrived at an overall effect size2. For studies with multiple effect sizes, only one effect size per study was included in the synthesis. For example, when a study reported multiple outcomes for the same “intervention vs. control group” comparison, the resulting multiple effect sizes were averaged within that study, when conceptually appropriate, to arrive at a single effect size. When a study involved more than two groups, which allowed for multiple comparisons, only the “intervention vs. control group” comparison was used to arrive at a single effect size for that study.3
Homogeneity Analysis
A homogeneity analysis examined whether the variation in a set of effect sizes may be attributed to sampling error alone or to other factors. Data synthesis was conducted using both a fixed and random effects models and report the results of both were reported but rely on the random effects model data was relied on to interpret findings. Under the random effects model, the estimates vary from study to study because of differences among the study population parameters (between-studies variation) and the sampling of different subjects within the study populations (within-study variation). Results from the random effects model allow for inferences to the population of studies from which the set was sampled in which case results of the data synthesis can be extrapolated beyond the studies included in the meta-analysis (Kline, 2004).
Sensitivity Analysis
We tested the robustness of our meta-analytic results to any one study in the meta-analytic portfolio of studies through a sensitivity analysis. The one study removed analysis assessed how the effect size changes with one study removed relative to the average effect size with all remaining studies included.
Publication Bias
To assess publication bias, we used the trim and fill procedure and visually inspected the resulting Funnel Plot (Rothstein, Sutton, and Borenstein, 2005).
Incomplete Reporting of Study Data
RCTs that met our inclusion criteria reported enough information to compute effect sizes. If information reported in an RCT was insufficient to compute an effect size, we attempted to contact the author(s). For example, we contacted the author of one study but the author was unresponsive. In a second instance, we were unable to locate the author or anyone who could assist us. In both cases, we excluded the studies from the analysis, rather than replacing the missing effect size value with zero which Pigott (2001) has shown to be an inappropriate option. This option tends to underestimate the effect size variance and can lead to incorrect inferences.
Post Hoc Subgroup and Moderator Analyses
As part of the meta-analysis, we conducted a sub-group and moderator analysis to examine differences in the average effect sizes by the following study-level characteristics: 1) Method of Parent Involvement and 2) Publication Type. Meta-Regression was used to assess the relationship between length of intervention and effect size.
Results
Descriptive Statistics for Included Studies
Nineteen RCTs met the inclusion criteria for the systematic review. One study, Phillips (1990), did not advance to the analysis stage because of a mismatch between the unit of assignment (schools) and the unit of analysis (students). Because outcomes were analyzed at the student level but the students were nested within the randomly assigned classrooms, their achievement measures were not independent. The remaining eighteen studies were published between 1964 and 2000 with the majority published as dissertations (67%). Most of the lead authors were female (83%).
The length of the parent involvement intervention programs ranged from 4 to 104 weeks with a mean of 23.2 weeks and a median of 10.5 weeks. The discrepancy between the mean and median was due to 1 study reporting 104 weeks of intervention. The most frequently reported method of parent involvement was collaborative reading (n = 8) followed by education and training (n = 4).
Of the studies that reported the socioeconomic status of the parents, 73% were from mixed SES, 9% were from middle SES and 18% were from lower SES. Of the studies that reported the ethnicity of the child, 25% were African American, 25% were Caucasian and 50% were of mixed ethnicities. Most of the intervention programs involved a mix of primary grades (47%) while 11% involved kindergarten, 11% involved first grade and 21% involved second grade; the remaining 10% were not reported.
All but one study was conducted in public schools (one study's school type was unreported) with twelve studies reporting a school location of suburban, urban, or rural. The majority (85%) were conducted in the United States while 10% were conducted in the UK and 5% were conducted in Canada. Most studies exhibited no differential attrition (70%).
Meta-Analysis of Included Studies
General Effects of Parent Involvement
We first assessed the general effects of parent involvement without respect to particular outcome domains. That is, we answered the general question, “Does PI result in an improvement in children's academic performance?”
This analysis suggests that parent involvement intervention programs of the kind reviewed here have a positive and significant effect on student achievement at the global level. Figure 1 shows the average effect of parent involvement on elementary school-age children's achievement across the domains of reading, math and science under fixed and random effects models. Figure 1 illustrates that this average effect was d= 0.43 using a fixed effects model.
When the 95% confidence interval for this effect size is considered, we observe that the average effect of parent involvement on achievement ranges from d= 0.30 to 0.56 and is statistically significant. This means that academic performance of children in the parent involvement group was approximately half a standard deviation higher than the academic performance of children in the control group.
A test of heterogeneity for the average effect size was applied to asses the amount of variability in study effect sizes beyond sampling error. The test produced a significant Q value of (Q = 35.6, df 17, p < .01) which means the variability in the effect sizes can be attributed to factors other than sampling error alone.
Another way to express the heterogeneity of effect sizes is by using an I2 value to quantify variation in effect sizes beyond sampling error. Under the fixed effects model, 52% of the variability of the effect size was beyond what was expected from random error (i.e., within study sampling of subjects). Thus, all subsequent results are reported using the results from a random effects model although we present effects sizes from both types of models so the reader can compare the results from both.
Under the random effects model, the overall achievement effect of parent involvement increased slightly to d=0.45 as did the confidence interval (95% CI =.025 to 0.66). These results further support the positive and significant impact of parent involvement on children's academic performance across achievement domains.
Publication Bias
Figure 2 shows that the 18 studies conform to the shape of the funnel plot are clearly symmetrically distributed around the vertical line that divides the funnel plot in half. This symmetry suggests the absence of publication bias. Further, when we applied the Trim and Fill analysis to these data (which trims excessively large studies and imputes small studies that are allegedly missing), the recomputed average effect size, displayed as the second diamond at the bottom of the funnel plot, does not change. These analyses show that the observed effect size of d=0.45 is based on an unbiased set of studies.
Sensitivity of Average Effect Size to One Study
A sensitivity analysis is important because it tells whether the positive average effect size of parent involvement was skewed by an exceptionally large effect size from any one study. For example, in the Heller (1993) study we found an effect size of 1.5. This effect size is three times as large as the average effect size of d=0.45 for all eighteen studies. Using the one study removed analysis, we assessed the average effect size and its corresponding confidence interval when each individual study's effect was removed from the average effect size calculation.
Figure 3 illustrates that the removal of any one study from the average effect size computation has little impact on the resulting effect size point estimate and its associated confidence interval. For example, Figure 3 also illustrates that when the Heller (1993) study is removed from the meta-analysis, the average effect size reduces to d=0.38 (95% CI = 0.26 to 0.53). Removing the Powell-Smith study which had the largest negative effect size increased the effect size slightly to d=0.45 (95% CI = 0.32 to 0.58). This analysis suggests that the average effect of d=0.45 on children's academic performance is not substantially impacted by one particular study.
Cumulative Analysis of the Effect of Parent Involvement
The results from the meta-analysis up to this point show that effect of parent involvement on achievement is positive, statistically significant, and stable with respect to the removal of any one study. The 18 studies in the meta-analysis embody 40 years of research resulting in an average effect of parent involvement on children's academic performance of d=0.45. We were intrigued by the following question: “In what year would we have discovered that the cumulative effect of parent involvement was positive if we had tracked the reporting of these studies?” To address this question, we conducted a cumulative analysis which adds the effect and sample size of the second study to the first. The effect of each subsequent study (i.e., the third, fourth, fifth, and so on) is then added to the cumulative effect (and sample size) of the preceding studies until all study effects and sample sizes have been aggregated.
Figure 4 presents the results of the cumulative analysis. Figure 4 reveals that if researchers had tracked effect sizes of RCTs on parent involvement since the first study was reported in the literature in 1964, they would have discovered that by 1971 the cumulative effect of parent involvement was positive and statistically significant. This effect has remained unchanged for more than three decades.
We turn now to examine the effect of parent involvement on the specific areas of reading, math, and science achievement.
Specific Effects of Parent Involvement
Reading Achievement
The achievement outcome most frequently reported for the included studies was reading which was assessed using norm and criterion referenced instruments. Results, summarized in Figure 5, are similar to those previously discussed for the effects of parent involvement on achievement in general. The average effect of parent involvement on reading achievement is positive and slightly larger than d=0.42 and statistically significant (95% CI = 0.18 to 0.66).
How sensitive was the average effect size for reading to the average effect size for any one study? Figure 6 shows that the average effect size is robust to the removal of any one study's effect size. The one study removed analysis produced effects within the range of d=0.37 to 0.49. In addition the confidence intervals associated with these effect sizes remained constant across the analysis.
Math Achievement
We reviewed RCTs that test the effects of parent involvement programs on children's math performance. Figure 7 shows that the point estimate for the average effect size in math achievement was d=0.54, borderline statistically significant, and substantially less precise (95% CI =0.02 to 1.07) than effect sizes for achievement in general and reading achievement in particular. The average effect size in math achievement reflected a substantial amount of heterogeneity in study effect sizes (I2 = 67.9%).
Figure 7 also shows that one study, Heller (1993), exhibited an effect size of d=1.50 (95% CI = 0.88 to 2.11) that is substantially larger than the effect size for each of the other four studies. When the effect size for Heller (1993) is Windsorized, Figure 8 shows that the average effect size is reduced to d=0.34.and is statistically significant (95% CI = 0.06 to 0.62).
Science Achievement
Only one study included in this review, Kosten (1997), focused on children's performance in science and revealed a d=.075 effect size (95% CI= -0.57 to 0.72). While we cannot generalize the results, there is a statistically non-significant effect of parent involvement on children's science performance.
Post Hoc Subgroup and Moderator Analyses
The results reported to this point enabled us to assess the main effect of parent involvement on achievement in general as well as specifically for reading, math and science. The effect of parent involvement may, however, vary by study characteristics such as the method of parent involvement. In other words, they may be interaction effects (at the study level) that might be uncovered through a planned, post-hoc subgroup analysis. Only three potential moderator variables provided an opportunity to unpack for whom parent involvement make the most difference: Approach of Parent Involvement, Length of Parent Involvement, and Publication Type.
Parent Involvement Intervention Programs
We examined average effect size varied by type of parent involvement for the 18 included studies. Figure 9 shows that intervention programs in which parents provided some kind of reward or incentive for their child's academic performance in math produced the largest average effect of d=1.18 (95% CI = .56 to 1.79).
Intervention programs in which parents were provided with education and training to improve their child's general academic performance produced the next largest positive and statistically significant effect size of d=0.61 (95% CI = 0.25 to 0.97) followed by studies in which parent's played reading games with their child.
Length of Intervention Program
A meta-regression analysis is presented in Figure 10 and shows there is no evidence of a relationship between the length of parent involvement and effect size (
Publication Type
This review focused on two types of publications: Grey Literature and Journal Publication. The Grey Literature included Dissertations and unpublished reports and comprised 67% of the included studies. Figure 11 shows a summary of the study effects for each publication type, in which parent involvement was compared to a control group. The summary of those studies in the grey literature yielded an effect size of d=.36 (CI 0.12 to 0.61) for the Grey Literature while those reported in journal publications yielded an effect size of d=.63 (95% CI 0.29 to 0.98).
Summary
Effects of Parent Involvement
Implications for Policy and Practice
This review concluded that there was substantial support for the positive impact of parent involvement with children outside of school to improve children's academic performance in school. The evidence suggested that the greatest impact of parent involvement on children's academic performance is in reading. A subgroup analysis indicated that the method of parent involvement produced positive and significant effects for some strategies (Education and Training and Rewards and Incentives) but not others. The positive effect of parent involvement on children's academic performance was achieved with parent involvement methods implemented between 6 and 28 weeks.
Implications for Research
There is a need to improve the quality of RCTs on parent involvement through adequate funding so that researchers can hire and train a sufficient number of research staff, recruit participants, and conduct fidelity of intervention and RCT implementation checks. There is a need to replicate high quality RCTs identified in this review as producing unusually large and significant effects such as the Heller and Fantuzzo (1993) and Fantuzzo, Ginsberg, and Davis (1995) studies. Although the evidence from this review pointed to positive and significant effects measured immediately following an intervention program. There is little evidence on the sustainability of these effects. A research policy is needed that fosters a research agenda on parent involvement that results in a balanced production of studies focused on the method of parent involvement, domain of achievement, and participant characteristics.
Implications for Reviewers
Review Limitations
Review Maintenance
In accord with the IRMG Policy Brief, the authors will examine the review every 3 years for update. Reasons for updating or not updating the review will be documented and forwarded to the Education Coordinating Group liaison to the IRMG (as recommended in the IRMG Policy Brief),
Responsible Reviewer(s) for Update of Review
The first author will be responsible for updating the review
Acknowledgments
The authors would like to acknowledge Anne Wade, Lesley Grayson, and Alan Gomersall for their expert advice on information retrieval; Kelly Borradaile for assistance in organizing the manuscript; the Education Coordinating Group Co-Chairs for conference calls that refined our thinking about how to conduct a systematic review; students in our Systematic Review course at the University of Central Florida that did the same; Dorothy de Moya for editorial, financial and moral support; and Robert F. Boruch for his thoughtful editorial comments.
This protocol was begun while Chad Nye was a University of Pennsylvania/Campbell Collaboration/Robert Wood Johnson Post Doctoral Fellow at the University of Pennsylvania 2001-02.
Sources of Support
The Campbell Collaboration Secretariat provided a grant to complete editing of the final draft of the review.
Supporting Non-authors
None.
Statement Concerning Conflict of Interest
None known.
References
Tables
Methodological Adequacies
Studies Excluded from the Systematic Review
Reasons Studies Were Excluded from the Systematic Review
Documentation of Search Strategies for the Systematic Review
Figures

Heterogeneity Statistics for a Fixed Effects Model: Q =35.6, df = 17, p = 0.005, and I squared = 52.3.

Funnel Plot of Standard Error by Hedges's g

Heterogeneity Statistics for a Fixed Effects Model: Q =35.6, df = 17, p = 0.005, and I squared = 52.3.

Heterogeneity Statistics for a Fixed Effects Model: Q =35.6, df = 17, p = 0.005, and I squared = 52.3.

Heterogeneity Statistics for a Fixed Effects Model: Q =22.5, df = 11, p = 0.023, and I squared = 50.5%

Heterogeneity Statistics for a Fixed Effects Model: Q =22.5, df = 11, p = 0.023, and I squared = 50.5%

Heterogenity Statistics for a Fixed Effects Model: Q = 12.5, df = 4, p = 0.014, I squared = 67.9%.

Heterogeneity Statistics for a Fixed Effects Model: Q = 1.2, df = 4, p = 0.876, I squared = 0.0%.

Heterogenity Statistics for a Fixed Effects Model: Q = 35.61, df = 17, p = 0.005, I squared = 52.3%.

Regression of intervention length on Hedges's g

Heterogeneity Statistics for a Fixed Effects Model: Q =35.6, df = 17, p = 0.005, and I squared = 52.3%
Footnotes
1
The asterisk represents a wildcard term such that the “parent*” term will search for “parent” and “parents” and “parental.”
2
This weighting gives more precise studies (e.g., studies with larger sample sizes) greater weight than less precise studies in the average effect size.
3
There were some studies included in the review for which there were more than one intervention group in which case there were multiple “intervention vs. control” group comparisons within a study (that were eligible for inclusion in the synthesis). In these cases, the resulting multiple effect sizes were averaged within the study to arrive at a single effect size.
