Abstract
Linked article:
1.0 BACKGROUND
The role of parents has long been thought to be centrally important to the academic achievement of their children. However, this role had neither been analyzed nor systematically studied using an experimental design until the 1960's. The evaluation of the Head Start Program in the United States (Coleman, Campbell, Hobson, McPartland, Mod, Weinfeld, & York, 1966) fostered a national focus on outcomes related to parental involvement by suggesting a substantial relationship between parental involvement in their child's education and their child's success in academic domains. Subsequent studies have been presented which support the findings from Coleman, et al. (Duff & Adams, 1981; Henderson, 1987; 1988). Even so, other studies have reported either mixed or no significant differences between experimental and control groups when measuring the effect of parental involvement on student achievement (Griffith, 1996; Heller, & Fantuzzo, 1993; Henry, 1974; Keith, Reimers, Ferman, Pottenbaum, & Aubrey, 1986; Ryan, 1964; Searles, Lewis & Morrow, 1982).
Some of the discrepancy across studies relates to the nature of the data collection and research design. For example, some investigators have studied the relationship between parental involvement and child school success using direct observation (Arbuckle & MacKinnon, 1988), surveys, or questionnaires (Edwards & Warin, 1999). Other investigators have utilized a traditional experimental design to compare student performances across the randomly allocated groups (DeBaryshe, 1993; Woods, Barnard, & TeSelle, 1974).
Another potential source of discrepancies in findings across studies relates to the outcomes measured. Specifically, a variety of dependent variables have been reported in studies on parental involvement including reading achievement (Epstein, 1987; 1991; Tizard, Schofield, & Hewison, 1982; Trovato & Bucher, 1980; Walberg, Bole, & Waxman, 1980; Woods, Barnard, & TeSelle, 1974), math achievement (Fantuzzo, Davis, & Ginsburg, 1995; Heller & Fantuzzo, 1993; Morgan & Sorensen, 1999), and perceptual skills training (Garrison, 1977). Furthermore, differences in study findings and the explanatory conclusions often do not take into account other important factors that can affect the validity of study findings such as the reliability of scales and tests; controlling for important child background characteristics such as grade, age, socioeconomic status; and controlling for important parent background characteristics such as socioeconomic status, education and training (Reynolds, Weissberg, & Kasprow, 1992).
In addition, researchers have defined parent involvement inconsistently or so broadly that is difficult to measure. For example, one group of researchers defined parent involvement as parent participation in educational activities at both school and home (Christenson, Rounds, & Gorney, 1992). Epstein (1987) suggested that parent involvement is multi-dimensional and included: (1) home environment that supports learning, (2) communication on classroom performance, (3) active attendance at school activities such as PTA, (4) engagement and monitoring of home learning activities, and (5) participation in school-based decision making such as school committees.
Typically, the study of parental involvement in home-school environments has identified educational activities that ranged from participation in parent-teacher organizations and conferences to specific training in activities designed to enhance an academic skill such as reading or math proficiency. The vast majority of the studies on parental involvement have focused on elementary school age children (Keith & Cool, 1992). Differences in results between these studies often can be attributed to how parental involvement is conceptualized and defined. Most schools in the US and abroad have some component of parental involvement integrated into the day-to-day operations; thus differentiating the effects of parent involvement from other factors that effect student achievement can be very difficult. In addition to the many primary studies on the role of parental involvement on student achievement, several narrative summaries have been published over the past 20 years that have addressed the issue of parental involvement. Gordon (1977), for example, reviewed parental involvement programs and proposed a three dimensional model of parent involvement: (1) Parent Impact, (2) School Impact, and (3) Community Impact. Based on available data, Gordon found that out of 14 Parent Impact studies identified, ten showed a positive effect on student academic performance while four showed no effect. Gordon reported that none of the studies addressed the School Impact Model and only four addressed the Community Impact Model.
In a more comprehensive review, Christenson, Rounds, & Gorney (1992) examined over 160 manuscripts under the rubric of parental involvement that focused on family components of parental expectations and attributions, structure for learning, home affective environment, discipline, and parent involvement. They concluded that while there is a positive correlation between student achievement and parental components, there are intervening variables between parental involvement components and student achievement. In other words, the influence of other variables such as SES, ethnicity, or gender had not been adequately accounted for while studying the interactive nature of the intervention of parental involvement and student achievement.
The first meta-analysis on the topic of parental involvement was conducted by Graue, Weinstein, and Walberg (1983) who quantitatively synthesized 29 studies conducted from 1970–1980 that focused on parental involvement and student academic success. Graue, et al. assessed a wide variety of parent involvement studies without specifically measuring the effect of a particular type of parent involvement, such as joint book reading or homework, per se. They did measure factors related to quality of research design and instruction delivery, without reference to specific parental activity. Graue, et al. concluded that there was a significant advantage for those children whose parents exhibited some level of involvement in their child's educational program.
Other narrative reviews have included bibliographic searches and annotated bibliographies. All concluded that there was a positive result for children whose parents were involved in their academic learning (Bronfenbrenner, 1974; Fantini, 1979; Henniger, 1979). The more recent reviews on the topic found that a combination of parent involvement, both at home and at school, appeared to be the most effective for positive outcomes regarding academic achievement (Henderson, 1987; Kagan, 1984).
The weight of support for parental involvement as a mechanism for assuring improved academic performance is consistently positive, although a number of earlier of studies were more qualitative than quantitative. When studing the effects of parental involvement on student academic success, researchers have relied on retrospective surveys, correlational studies of existing conditions, and anecdotal observations of both parent involvement quality and student achievement. Graue, et al (1983) appears to be one of the few summary studies in which a quantitative assessment of parental involvement has been assessed as an intervention whose effect was measured by student achievement. Thus, the purpose of this review is to conduct a systematic review of parental involvement effects on student academic performance that will in part update the Graue, et al. study by summarizing, quantitatively synthesizing, and analyzing more recent studies that use experimental designs.
2.0 OBJECTIVE
To determine the effectiveness of parental involvement in improving the academic performance of elementary school age children in grades K-6.
3.0 METHOD
3.1 Information Retrieval Tasks
In following the policies and guidelines proposed in the Information Retrieval Methods Group (IRMG) Policy Brief (2004), we will conduct the following information retrieval tasks:
Electronic Searches of Databases, Hand Searches of Journals, Hand Searches of Grey Literature, Documenting Search and Selection Processes Contributing to C2's trials registers (C2-SPECTR and —PROT)
3.1.1 Electronic Searching Of Databases
We plan to search 18 well-known electronic databases identified in the IRMG's policy brief's “List of Databases of Possibly Relevant to C2 Reviews.” These databases are listed in Appendix A. If during our search we come across others that might be relevant to the review, we will search those also. The purpose is to cast a wide net such that our search is as comprehensive and unbiased as possible. From the list provided in the policy brief, we will select databases deemed relevant to the review
3.2 Search Strategies
For each database searched, we will use the following substantive terms to locate relevant studies for the review:
parent∗ Involvement, or parent∗ participation, or parents-as-teachers, or parent∗ effectiveness, or parent∗ effectiveness training, or parent-child relationship, or parent∗ education
reading, or literacy, or math∗, or writing, or spelling;
child∗, or pre-school, or preschool, or school age∗, or school-age∗, or elementary school, or elementary-school, or elementary grades, or elementary-grades
We will consult database thesauri when they are available to ensure that the universe of appropriate synonyms have been included in the intervention, outcome, and target population search term categories. Search terms will be modified to meet the requirements of individual databases. When possible, we will include studies reported in languages other than English.
3.2.1 Hand Searches of Journals
Page-by-page hand searches of the entire text of journals and other literature are an integral part of locating as many trials as possible for inclusion in systematic reviews. Although visual inspection of hardcopy text is the most common interpretation of what is meant by performing a “hand” search, recent technological advances permit visual inspection of full-text articles available in many online journals (Rothstein, Turner, and Lavenberg, 2004). Care must be taken, however, to verify the completeness of the online version.
Parental involvement is a interdisciplinary topic. As such, the topic is reported on in a wide variety of journals. There are probably many journals, rather than a few, that are relevant to this review. Based the results of our electronic searches, we will attempt to identify the journals most likely to yield relevant studies. If the journals on our priority list have not been hand searched by Campbell or Cochrane already—based on our review of the Cochrane and Campbell Master Lists of Journals being Handsearched—then we will hand search these journals for studies relevant to our review and for studies relevant for inclusion in C2-SPECTR (i.e., randomized controlled trials).
3.2.2 Hand Searches of Grey Literature
Previous research in the social sciences (Glass, 1981) and more recent research in the health sciences (Hopewell, Clarke, Lusher, Lefebvre, & Westby, 2002;) have shown that including grey literature in a meta-analysis may reduce bias. The amount of bias reduction will vary according to the amount and quality of the grey literature excluded from the review, and the domain in which the review was conducted (Egger, Juni, Bartlett, Holenstein, & Sterne, 2003). To the best of our knowledge, the What Works Clearinghouse (WWC) Literature Search Handbook and C2's IRMG Policy Brief, provide the most up-to-date guidance on searching the grey literature in the social and education sciences. Using these two documents as a guide, we will develop a search protocol to systematically search the following sources of grey literature: Government Reports Book Chapters Dissertations Research Organization's websites Personal Networks
3.3 Documenting the Search and Selection Processes
3.3.1 Electronic Searches
As proposed in the C2 IRMG Policy Brief, we will document our electronic substantive search strategy to locate studies for this review. This documentation will include: search engines used (e.g., Ovid), the main sources searched (e.g., PsycINFO), time period searched (e.g., 1966 to 2003), and notable features used during the search, such as language.
3.3.2 Hand Searches
After we decide on the journals to hand search for relevant studies, we will consult the Cochrane and Campbell Master Lists to ensure that we are not duplicating a previously conducted hand search. If the journal we plan to hand search is not on the lists, then we will formally register the hand search with the C2 Hand Search Coordinator.
A copy of the Table of Contents of each issue will be used as the primary hand search documentation. On the Table of Contents, we will classify each article in the issue with regard to its study design (i.e., an RCT). For each randomized or possibly randomized trial identified, a photocopy of the title, abstract, and text (usually in the methods section) upon which the study classification is based will be attached to the Table of Contents page.
In general, we plan to document all information retrieval activities in the review. These activities will be described in sufficient detail so that the process can be replicated by other researchers.
3.4 Contributing to C2's Trials Registers (C2-SPECTR and C2-PROT)
All citations of studies identified during the information retrieval phase of the review and meet the general criteria for inclusion in C2-SPECTR (i.e., the study is an RCT or systematic review with references to RCTs) will be denoted as such in an annotated bibliography. The annotated bibliography will include studies that were relevant to the specific review and those that were not. Studies not relevant to this review might be relevant to other C2 reviews and as such are relevant for inclusion in C2-SPECTR. The annotated bibliography will also include articles that are ancillary to studies relevant for inclusion in C2-SPECTR such as reanalysis, editorials, letters to the editors, and the like. As recommended in the IRMG Policy Brief, this bibliography will be forwarded to the Education Group's Assistant Coordinator who is responsible for cataloging studies in a specialized register that supplies studies to C2-SPECTR.
3.5 Selection Criteria for Trials
3.5.1 Study Criteria
Only studies that measured the following dimensions of the intervention and child outcomes will be included in this review: Parents involvement with their child in academic support activities outside of school (e.g., reading or completing supplemental math problems with the child); Parents involvement (as defined in “a.”) for a minimum of 20 days; Child's academic performance in reading, mathematics, spelling, or writing
3.5.2 Design Criteria
We will include only Randomized Controlled Trials (RCTs) in this study. Studies that utilize solely single group, single subject, or qualitative approaches will not be included in the review.
All included studies will identify both experimental and control (or comparison) groups to which participants will be assigned. A control group is defined as a non-treatment condition while a comparison group is defined as an alternative treatment condition.
A description of the included study designs follows.
3.5.2.1 Randomized Controlled Trial (RCT)
An RCT is a trial in which the participants were definitely assigned prospectively to one or more alternative forms of intervention using a process of random allocation such as a random number generation or coin flips.
Retrieved studies will be screened for topic relevance and research design quality based on the title and abstract first. If, the study's topic relevance or research design appropriateness cannot be ascertained from the title or abstract, we will retrieve the full text version of the study.
Once all potentially topic-relevant and research-design appropriate studies have been retrieved, two reviewers will evaluate each study independently for inclusion in the review. If there is a disagreement on the inclusion of a study, two reviewers will attempt to identify the source of the disagreement and reconcile to a final decision. If a final decision cannot be reached, the full-text version of the study will be submitted to the third reviewer for a decision. Reviewers will not be blinded at any level of review to the name(s) of the author(s), institution(s) or publication source.
3.6 Coding Categories for Study Quality
3.6.1 Study Characteristics
3.6.1.1 Assessment of methodological quality
Two reviewers will independently evaluate each study for its methodological quality according to design, participant, and intervention characteristics.
3.6.2 Design Characteristics
3.6.2.1 RCT Design Studies
We will evaluate the following characteristics of included randomized controlled trials and controlled clinical trials in the review based on the following criteria if reported (Newman, 2001): Method of random assignment Protection against student exclusion bias Blinded assessment of primary outcome(s) Pre-treatment measurement used Reliability of primary outcome measure(s) Protection against contamination
3.6.2.2 Attrition
In addition to evaluating each study for methodological quality, two reviewers will evaluate each study for attrition rates. Any study with more than 30% pre- to post-treatment measure attrition will be excluded.
3.6.3 Participant Characteristics
Each study must report data on a minimum of five participants in both experimental and control (or comparison) groups. The minimum number of group participants was selected in order to maintain the interpretability of the mean and standard deviation that will be used in effect size and standard error calculations. Specific participant characteristics for both the experimental and control (or comparison) groups will be coded to include the following: Age Grade Gender Number of Participants SES Ethnicity
3.6.4 Intervention Characteristics
All studies will be coded for intervention characteristics as described in the following sections (3.8.1 – 3.8.3)
3.6.4.1 Length of Intervention Program (in days)
All intervention programs must have been implemented for a minimum of 20 school days to be included in this review.
3.6.4.2 Type of Parent Involvement
All interventions must involve planned and consistent direct contact between the parent(s) and child in an educational activity outside the school-day setting. Studies in which parents are engaged in a classroom activity as an aide or support personnel will be excluded from the review. In addition, interventions that measure parent involvement for participation in school-related activities such as parent teacher organization meeting, school program attendance, or home visits will also be excluded from the review. Examples of types of parent involvement that will be focus of this review include: Reading (with child) Solving math problems (with child) Essay writing (with child) Homework assistance Study Skill Instruction Exam Preparation
3.6.4.3 Types of Outcome Measures
Outcome measures for each intervention approach must be a quantified assessment of educational performance using formal standardized tests, criterion referenced tests, or rating scales of academic performance, with appropriate psychometric properties.
3.7 Description of Methods Used In Primary Research
The most common method used in parent involvement research is to compare the implementation of some type of parent involvement with their child against a comparable group of parents and their children without this involvement, or with a different type of involvement. In some studies there will be a comparison between two or more types of involvement in addtion to a no-treatment group serving as the control.
Most studies that we expect to locate will provide multiple measures of child performance such as standardized achievement test scores, number of correct answers, proportion of correct production to target, or parent perception of changes in child performance. These outcomes are usually treated as dependent variables. Independent variables usually include children background characteristics, parental background characteristics, length of parental involvement, and frequency of parental involvement.
3.8 Criteria for Determination of Independent Findings
In the case where a single study has multiple outcomes, we will employ a shifting unit of analysis approach. When estimating the overall effect size of an intervention, the study is represented by the mean value of all outcomes in the study. However, when examining potential moderators of the overall outcomes, a study's results will be aggregated only within the separated categories of the moderator variable(s). For example, if a study on the effect of parental involvement on reading achievement measured two outcomes, reading speed and reading comprehension, those two effects would be averaged for purposes of estimating the intervention's effect on reading. However, when examining the type of outcome measure as a moderator variable, the study would contribute an effect size to the “reading speed” category, and an effect size to the “reading comprehension” category.
4.0 STATISTICAL PROCEDURES
4.1 Data Extraction
Two reviewers will extract data from the articles independently using a form to be developed to code methods, participant characteristics, intervention characteristics, and outcomes. Uncertainty and disagreement will be resolved through discussion and consultation with a third reviewer. If further information (e.g., missing data) is required regarding study data in order to conduct appropriate analyses of outcomes, the first author of the study will be contacted. Should requested data be unavailable the study will be reported but not included in the final analysis. Information extracted from the studies will be displayed in an included studies table of the review that displays included and excluded studies.
4.2 Data Synthesis
4.2.1 Effect Size Computations
The choice of which effect size computation to use depends on three key factors: (1) the measures of the outcome variable(s), (2) the designs of studies being reviewed, and (3) the statistical analyses that have been reported. Depending on the study results, we will compute effect sizes from one of three families described next.
4.2.1.1 Mean Difference Statistic (d-index)
Studies reporting continuous outcomes will be summarized using standardized mean differences where different measures have been used to measure the same outcome. When the outcome has been measured using the same measure, a weighted mean difference will be used.
4.2.1.2 Odds Ratio Family (including risk difference and risk ratio)
Studies reporting binary data in which mean outcomes are compared in the experimental and control (or comparison groups) will be summarized using odds ratios with 95% confidence intervals for the odds ratio, risk difference, or risk ratio statistics.
4.2.1.3 Standardized Regression Coefficient (including the correlation coefficient r)
This metric will be used when studies have a continuous outcome measure, the study designs assess the relation between a quantitative predictor and the outcome, and the analysis uses regression or General Linear Models (GLM).
4.2.2 Converting Effect Sizes
We will convert all effect size indices to Hedge's g which is a standardized mean difference with a small sample size bias correction factor.
4.2.3 Combining Effect Sizes
Where possible, effect sizes will be averaged across studies by using an inverse variance weighting of the individual effect size to account for differences in sample sizes for individual studies. This weighting will result in the individual effect sizes of larger n studies being given more weight in the combined effect size.
4.2.4 Homogeneity Analysis
A homogeneity analysis will examine whether the variation in a set of effect sizes may be attributed to sampling alone or if other sources contribute to the variation. Data synthesis will be carried out using a random effects model, which assumes that the effect size in each study is a sample estimate from a different population, and it estimates the parameter for that population. The estimates differ from study to study owing differences among the study population parameters (between-studies variation) and to sampling of different subjects within the study populations (within-study variation). More important, from a policy perspective, results from the random effects model allow for inferences to the population of studies from which the set was sampled. In this case, results of the data synthesis can be extrapolated beyond the studies in the set.
4.2.5 Sensitivity Analysis
Through a sensitivity analysis, we will test the robustness of the conclusions drawn from the analysis. In most, if not all, meta-analysis, there is the potentially biasing influence of study characteristics such as attrition, type of treatment, missing data, sample size, study design, or publication source. To evaluate the possibility of such bias, we will use funnel plots to assess the relationships between effect size and study precision. Such a relationship could be due to identifiable biases or to less obvious systematic differences between studies. If a relationship is identified, the instructional diversity of the studies will be further examined for possible explanations (Egger, 1997).
To increase confidence in the robustness of the results of the data synthesis, we will potentially conduct five sets of sensitivity analyses aimed at examining whether the results are sensitive to the inclusion or exclusion of particular studies and whether results are sensitive to the method of analysis.
4.2.4.1 Participant Attrition
Studies with at least 70% of participants at posttest/follow-up will be examined separately and the results compared descriptively to those studies with greater than 30% attrition. Overall attrition greater than 30% reduces the statistical power and precision of the estimate of effect of the study thus reducing the ability of detect treatment effects when they exist (Boruch, 2004).
4.2.4.2 Efficacy versus Effectiveness Trials
Studies where control condition participants received an alternative intervention of parent involvement as opposed to no parent involvement will be examined separately.
4.2.4.3 Publication Bias
Mean effect sizes of studies retrieved from peer reviewed sources will be compared to mean effect sizes of studies retrieved from unpublished sources (e.g., dissertations, government reports, and conference presentations). We will consult with the C2 Statistics Methods Group to determine if the are other methods that can be used to statistically assess publication bias.
4.2.4.4 Study Design
As stated earlier in our conclusion criteria, we will include only RCTs in the review.
4.2.4.4 Missing Data
The first authors of primary studies will be contacted and asked to supply missing data. If the author cannot be contacted or data cannot be supplied, missing data and dropouts will be assessed for each individual study. For studies in which effect sizes cannot be computed, the effect sizes will be set to zero. Mean effect sizes for all studies (including those with missing effect sizes that were set to zero) will be compared to mean effect sizes for the subset of studies with no missing data. Depending on the number of studies with missing data, we will consider using Multiple Imputation methods to impute estimates of missing values in the sensitivity analysis.
4.2.5 Sub-Group Analysis
The impact of certain study characteristics will be examined through sub-group analyses such as: The role of student age or grade in the effectiveness of parent involvement, The role of parent SES in the effectiveness of parent involvement, The effect of parent involvement on the different areas of achievement measured such as reading and math, The role of parents in the effectiveness of the intervention, The length of the intervention.
5.3 Treatment of Qualitative Research
Qualitative research will not be included in this review.
5.0 REVIEW MAINTENANCE
In accord with the IRMG Policy Brief, the authors will examine the review every 3 years for update. Reasons for updating or not updating the review will be documented and forwarded to the Education Coordinating Group liaison to the IRMG (as recommended in the IRMG Policy Brief),
5.1 Time Frame for Protocol Completion
Once the Protocol is approved, we anticipate submitting the first draft of a completed review to the Education Group within six months.
5.2 Responsible Reviewer(s) for Update of Review
The senior author will be responsible for updating the review
6.0 Acknowledgements
This protocol was begun while Chad Nye was a University of Pennsylvania/Campbell Collaboration/Robert Wood Johnson Post Doctoral Fellow at the University of Pennsylvania 2001–02.
6.1 Sources of Support
None.
6.2 Supporting Non-authors
None.
7.0 Statement Concerning Conflict of Interest
None known.
Footnotes
1 The asterisk represents a wildcard term such that the “parent∗” term will search for “parent” and “parents” and “parental.”