The relationship between teacher qualification and the quality of the early childhood education and care environment

Defining the quality of early childhood education

Katz (1992) suggests that the quality of ECEC can be defined and assessed from four perspectives: top-down; bottom-up; inside; and outside-inside. Assessment based on the top-down perspective examines quality primarily by identifying selected features and characteristics of the program and environment, as recorded or observed by an assessor. The bottom-up perspective gives weight to the children's experience of the program and considers to what extent the young children find themselves being included and respected during the learning process. This approach is especially applicable for ECEC programs that focus on the inclusion and integration of children with developmental or learning delays and disabilities into the environment (Spiker, Hebbeler, & Barton, 2011). The inside perspective defines quality of an ECEC program as perceived by the staff. Three dimensions are considered; collegial relationships, staff-parent relationships, and staff-sponsor relationships. The outside-inside perspective focuses on the parent-teacher relationship. This perspective may reflect whether the parents’ expectations or pedagogical goals are compatible with the curriculum. It is also suggested that the learning environment for children is more effective when parents are meaningfully engaged in the program through the teacher or service provider (National Association for the Education of Young Children, 1991).

Defining quality in ECEC has been a continuing challenge in terms of the methodology in research and formulation of policy to enhance quality (La Paro, Thomason, Lower, Kintner-Duffy, & Cassidy, 2012). Among the varying definitions or conceptualizations of quality in reference to ECEC, the predominant approach is the top-down perspective (Katz, 1992). Traditionally, the connotation involves both multiple distal (e.g., broad parameters of program and state policies) and proximal (e.g., curriculum and caregiver-child interactions) features of an ECEC environment that are expected to support children's development in various domains (e.g., cognitive, language and social development) (Dunn, 1993; Kontos, 1991). The relative importance of indicators of either proximal or distal features varies across a multitude of perspectives, including that of the parent, caregiver and child (Layzer & Goodson, 2006). In order to avoid vague and nonspecific operational definitions of quality (e.g., any “all-encompassing” term), early childhood researchers have commonly conceptualized and disaggregated the multidimensional ECEC quality into two measurable interrelated components: (1) structural quality - this refers to structural indicators such as child-to-staff ratios and caregiver characteristics such as teacher formal education; and (2) process quality - including learning opportunities available to the children, and teacher-child and peer-to-peer interactions within the child care environment (Clifford, Reszka, & Rossbach, 2010). Both qualities generally share the same goal (or target), which is the enhancement of child development and learning outcomes (Taguma, Litjens, & Makowiecki, 2012). Previous literature reveals that both structural and process characteristics of quality are not only related, but are also important to children's developmental outcomes (Sammons et al., 2002).

Structural quality

A number of studies have examined the effectiveness of structural features to predict quality outcomes. Three frequently studied features are child-to-staff ratio, class size, and staff educational attainment. Children enrolled in programs with low ratios (i.e., few children per caregivers) tend to be more socially competent, compliant, cooperative and advanced in academic progress compared to children from programs with high ratios (Clarke-Stewart & Gruber, 1984; Guo & Harris, 2000; Howes, 1988; Whitebook, 1989). High child-to-staff ratios are also more frequently associated with negative staff-child interactions (Rosenthal & Vandell, 1996). Smaller group sizes (compared to larger group sizes) are associated with better academic progress, higher levels of cooperation, compliance, consideration and engagement in sophisticated social play. Children in smaller groups also have a lower tendency to answer with antisocial responses during social problem-solving tasks (Clarke-Stewart & Gruber, 1984; Dunn, 1993; Howes, 1988; Whitebook, 1989). When caregivers have higher education levels, children tend to have higher levels of social competence (Clarke-Stewart & Gruber, 1984). Education or training of teachers/caregivers in the ECEC environment is also correlated with higher scores on achievement tests, and higher levels of compliance and cooperation in children (Whitebook, 1989).

It is noted, however, that structural indicators of an ECEC program and caregiver characteristics contribute to the overall quality or child developmental outcomes through relatively indirect ways compared to process features (Vandell & Wolfe, 2000). For example, Blau (2000) proposes a two-stage model, suggesting that structural features contribute to the process quality in ECEC settings during the first stage, which subsequently affects children's developmental outcomes. Similarly, Phillipsen, Burchinal, Howes and Cryer (1997) suggest that the effects of structural variables on child developmental outcomes are mediated by process quality (e.g., more nurturing and extensive teacher-child interactions).

Process quality

Process quality is found to be more predictive of children's learning and developmental outcomes than structural indicators (Vandell & Wolfe, 2000; Whitebook, 1989). The classroom dynamics, including proximal-level interactions and transactions among teachers, children and materials, reflect the core components within the early childhood learning environment. These components are associated with positive academic and social outcomes for young children (Love, Meckstroth, & Sprachman, 1997). A higher process quality classroom tends to be associated with fewer problem behaviours, better cognitive and social skills, and academic progress - especially math skills (Peisner-Feinberg & Burchinal, 1997; Peisner-Feinberg et al., 2001). The concept that higher process quality supports better educational, social and emotional outcomes is also supported by the developmental prevention literature. This literature highlights that developmental and social experiences vary between groups (e.g., those living in high or low socio-economic status areas) and that these differences in experience may be affected by the environment in which the child grows and learns (Blokland & Nieuwbeerta, 2010; Homel, 2005). As such, ECEC institutions, together with the involvement of parents and caregivers, play a pivotal role in the social, emotional and educational development of the child (Sylva et al., 2006; Warash, Ward, & Rotilie, 2008). Further, these institutions need to move beyond traditional system silos where there may be common goals between institutions, but coordination is poor or non-existent. In this regard, it has been the priority of policy makers, and education and developmental experts to monitor and enhance process quality in the ECEC sector.

Measuring quality

Process quality is assessed primarily by observing the early childhood learning environment (Clifford et al., 2010). In some instances, the measures focus on specific aspects of the teacher-child interaction (e.g., the Caregiver Interaction Scale (CIS; Arnett, 1989), and the Student Teacher Relationship Scale (STRS; Pianta & Nimetz, 1991)). The Classroom Assessment Scoring System (CLASS) is one of the most widely used and reliable measurement tools to assess the quality of classroom interactional processes. Specifically, CLASS focuses solely on several dimensions of teacher-child interactions (Pianta, La Paro, & Hamre, 2007). These dimensions are organised into three broad domains (emotional support, classroom organisation and instructional support) reflecting two aspects (social and instructional) of interactions (Hamre et al., 2012; Mashburn et al., 2008). The social aspects of interactions focus on how sensitive and responsive teachers are with respect to children's needs and cues. The instructional features of interactions focus on teachers’ behaviours that promote children's development and performance of skills. Hamre and Pianta (2005), who examined the influence of support from teachers on children's school readiness, found that high levels of instructional and emotional support reduce the probability (for children at risk of early school failure) of developing poor learning outcomes and reduce children's conflict with teachers. Mashburn and colleagues (2008) also discover that instructional interactions are able to predict academic and language skills of children, and teachers’ emotional interactions are able to predict teacher-reported social skills.

Environment rating scales

The entire notion of the learning environment, however, is beyond the specific interactional aspects as measured by CIS, STRS and CLASS, and is defined in a broader sense with interactions between program components and among people (e.g., teacher, parent and child) in the ECEC setting (Harms & Clifford, 1983). Hence, several Environment Rating Scales (ERS), as the global measure of ECEC quality, have been developed. They include the Early Childhood Environment Rating Scale (ECERS) (Harms & Clifford, 1980), the Infant/Toddler Environment Rating Scale (ITERS) (Harms, Cryer, & Clifford, 2003) and their revised versions (ECERS-R (Harms, Clifford, & Cryer, 1998) and ITERS-R (Harms et al., 2003)). The ERS focus on multiple processes within the early childhood learning setting and are recognised as a set of standardised measurement tools that are widely used in research assessing ECEC quality (Clifford et al., 2010). Subscales of the ERS include: program structure (i.e. focusing on the schedule, time for free play, group time and provisions for children with disabilities); activities (i.e. focusing on the provision and quality of activities including fine motor, art, music, dramatic play and math/number); language and reasoning (i.e. focusing on the formal and informal use of language, development of reasoning skills and communication); parent and staff (i.e. focusing on the provisions for personal and professional needs of staff and parents, and staff interaction and cooperation); space and furnishing (i.e. focusing on the quality of items including indoor space, furniture for routine care, room arrangement and space for privacy); interactions (i.e. focusing on discipline as well as supervision and facilitation of proper interactions between children and staff and among children); and personal care routines (i.e. focusing on teaching and practice of routines including greeting/departing, meals/snacks, toileting/diapering, health and safety). The ERS, when compared to other commonly used measures (e.g., CLASS), focus not only on items related to teacher-child interactions, but also on available materials, and health and safety issues (Hamre, Goffin, & Kraft-Sayre, 2009).

ECERS and ECERS-R have, since their development, been the primary measurement tool for researchers and professionals in the ECEC field (Tout, Zaslow, Halle, & Forrey, 2009). Although other measurement tools are available that assess the early childhood learning environment (e.g., the Observational Rating of the Care Environment (ORCE) Qualitative Ratings (NICHD Early Child Care Research Network, 1999)), they tend not to be widely applied. This is also the case for the Supports for Early Literacy Assessment (SELA) tool (Committee on Developmental Outcomes and Assessments for Young Children, 2008), which, to date, remains underdeveloped.

Halle, Whittaker and Anderson (2010) produce a compendium of measures, reviewing some 50 instruments, which measure the quality in ECEC settings and compare them in terms of target age, purpose (e.g., accreditation and evaluation), method (e.g., observation, interview and document), child developmental domains (e.g., language development, literacy and general cognition), structure (e.g., family involvement, activities/scheduling, classroom organization and materials), administration (e.g., internal communication and leadership/management), improvement (e.g., professional development and program/staff assessments) and training and administration (e.g., cost).

According to the ‘cross-cutting’ tables created by the authors, ECERS-R is the most comprehensive measure that assesses most of the quality features within ECEC environments reflecting on child development, administrative structure and staff development. With respect to the quality features that support school readiness and subsequent academic progress, ECERS-R covers eight out of nine domains including language development, literacy, math, science, creative arts, social and emotional development, approaches to learning, and health/physical development. General cognition is not measured by ECERS-R. For quality features related to structure, administration and staff, ECERS-R covers four out of five domains under the structure category, all domains under the administration category, and two out of three domains under the monitoring and improvement category. The two domains that are not measured by ECERS-R are business practices and assessments/monitoring of students.

Both the breadth of multifaceted information measured by the ECERS-R and the general lack of supplemental measurement tools contribute to the predominant use of ECERS-R as the primary scale for assessing quality in ECEC settings (Tout et al., 2009). As a pioneering self-assessment tool in ECEC, the ECERS-R has been used in recent large studies of classroom quality such as the National Centre for Early Development and Learning's (NCEDL) Multi-State Study of Pre-Kindergarten and Study of State-Wide Early Education Programs (SWEEP) (La Paro et al., 2012). Further, ECERS-R has gained additional influence on practice and policy-related decisions in the field, as it has become the measure of quality in the state-wide Tiered Quality Rating and Improvement Systems (TQRISs) in the United States (Tout et al., 2009). In the State of California, for example, the ratings produced by ECERS-R have been used for the selection of mentor teachers in 70 community colleges. California's Compensation and Recognition Encourage Stability (CARES) program also uses ECERS-R to assess teachers continued participation.

The first editions of ECERS and ITERS have been used as comprehensive quality measures in nation-wide studies. For example, the ECERS and its revised version (ECERS-R) are used in the Head Start Family and Child Experiences Survey (FACES) study in the U.S. (Espinosa, 2002). Both ECERS and ITERS were used in the National Child Care Staffing Study (Whitebook, 1989), and the Cost, Quality, and Child Outcomes Study (Helburn et al., 1995). Clifford and colleagues (2010) note that when information is not available for the revised version (i.e., ECERS-R and ITERS-R), the first edition of the scale is suitable. Evidence confirms that data from studies using ECERS are directly comparable to data from studies using the ECERS-R (Sakai, Whitebook, Wishard, & Howes, 2003). Further, the authors of the Environment Rating Scale (ERS) have intended to make ECERS, ITERS and their revised versions as comparable measures of quality (Sakai et al., 2003). Therefore, the scoring and formatting of ITERS and ITERS-R are consistent with that of the ECERS-R. Hence, for the purposes of this systematic review, quality of the ECEC environment will be assessed on studies that have used as their measurement tools, ECERS, ITERS and their revised versions (ECERS-R and ITERS-R).

Evidence of predictive validity of ECERS, ECERS-R, ITERS and ITERS-R

The scientific evidence linking process quality (as measured by the ERS) and child learning and developmental outcomes is compelling, although not unanimous. Some researchers (e.g., Gordon, Fujimoto, Kaestner, Korenman, & Abner, 2013; Gordon, Hofer, Fujimoto, Risk, Kaestner, & Korenman, 2015) argue that the predictive validity of environment rating scales is not exceptionally consistent. Gordon et al. (2015) used the item response theory (IRT) analyses to reveal that although every indicator (254 ECERS-R indicators as identified by the experts) was generally relevant to child care quality and relevant to at least one of the three meta-domains (i.e., cognitive, socioemotional, and health) and at least one of the eight subdomains (e.g., promote math skills, promote social competence, reduce injuries), there was little concensus with regard to which domains the attribute supported as indicators did not fit together to measure single dimensions. These results were, however, limited as the study only relied on expert ratings of the relevance of indicators for aspects of child development and did not directly look at the indicators’ validity in relation to school readiness.

Regarding other researchers who suggest that the practicality of ECERS is sound and supported by evidence, Cassidy and colleagues (2005) argue that any unfavourable results, with respect to the predictive validity of ERS, are due to collapsing the multitude of features of ECEC environments into a unidimensional comprehensive index – omitting the specific mechanisms responsible for development (e.g., language reasoning, and space and furnishings). Scientific examinations of associations between the scores on ERS and children's learning and developmental outcomes, therefore, commonly involve the use of both overall and subscale scores of ERS. For example, Howes and colleagues (2008) decompose the ECERS-R into two distinct factors; language/interactions and physical environment. They discover that language/interactions factors are more predictive of children's development than physical environment factors alone.

Previous research (e.g., Sammons et al., 2003a) suggests that the quality of the ECEC environment is associated with children's learning and developmental outcomes in general. For example, Love and colleagues (2004), who measure child learning outcomes using the Bayley Scales of Infant Development Mental Development Index (BSID-MDI) (Bayley, 1993), discover that higher overall scores on ITERS/ECERS-R can predict higher scores on the intellectual and cognitive development screener. The screener covers several types of abilities including, for example, sensory/perceptual acuities, memory learning and problem solving, vocalization and beginning of verbal communication, mental mapping, complex language and mathematical concept formation. Furthermore, in view of the complexity and multidimensionality inherent in children's learning and developmental outcomes, some researchers have identified several outcome categories (e.g., mathematics and numeracy skills, language and literacy, and social outcomes) and developed measures to investigate the impact of ECEC programs on each outcome category.

ERS have been shown to be predictive of children's performance in standard measures of mathematic achievement and numerical skills. For example, Peisner-Feinberg and colleagues (2001), using a shortened version of the ECERS, discover a positive correlation of ECERS ratings with higher scores on the math achievement applied problems subset of the Woodcock-Johnson Psycho-Educational Battery - Revised (WJ-R) (Woodcock & Johnson, 1990). WJ-R is designed to measure cognitive abilities, scholastic aptitude, and academic achievement in three areas - reading, mathematics, and written language. A positive relationship between the social interaction subscale on the ECERS-R and children's early number concept development has also been revealed (Sammons et al., 2003a). A more recent study (Burchinal et al., 2008), using the ECERS-R as the measurement of quality in pre-kindergarten and kindergarten settings, shows that the Teaching and Interactions factor is predictive of children's performance on the WJ-R math achievement applied problem subset. Anders and colleagues (2012), using the arithmetic subscale of the Kaufman Assessment Battery for Children (KABC) (Melchers & Preuss, 2003), found a marginally significant relation between the total score on ECERS-R and children's numeracy levels over the preschool period (from the first to the third year of preschool).

The predictive validity of ERS has also covered children's language and literacy performance. Love and colleagues (2004) discover that children in ECEC settings with higher overall ECERS-R or ITERS scores tend to perform better in the Peabody Picture-Vocabulary Test – Third Edition (PPVT-III) (Dunn & Dunn, 1997). PPVT-III is a measure of receptive vocabulary for standard English and a screening test of verbal ability. Peisner-Feinberg and colleagues (2001), using the Peabody Picture Vocabulary Test – Revised (PPVT-R) (Dunn & Dunn, 1981), a former edition of PPVT-III, also discover an association between higher overall scores on a shortened version of the ECERS and children's scores on the PPVT-R. In addition, Bryant and colleagues (2003) note that children from higher quality preschool settings, as assessed by the ECERS, possess greater book knowledge and have higher print awareness scores on Zill and Resnick's (1998) Concepts About Print Assessment. Further, children's expressive language development, as measured by the Oral Expression Scale from the Oral and Written Language Scale (OWLS) (Carrow-Woolfolk, 1995), is shown to be related to the ECERS-R total score (Mashburn et al., 2008), and the Teaching and Interactions subscale (Burchinal et al., 2008). Burchinal and colleagues also discover a positive relationship between receptive language scores on the PPVT-III and scores on the ECERS-R Teaching and Interactions subscale. In summary, Clifford (2010) argues that higher quality environment, as measured by environment rating scales, is associated with children's development in three areas, including receptive language, print awareness, and book knowledge.

With respect to social outcomes, several important elements (e.g., social and behavioural development) can be predicted by using ERS (Sammons, 2010). Sammon and colleagues (2003b), who focus on four measures of social/behavioural development in Goodman's (1997) Strengths and Difficulties Questionnaire (i.e., self-regulation, hyperactivity, pro-social behaviour and anti-social behaviour), discover that the scores on the caring and emotional/relationship aspects of ECERS-R is associated with reduced hyperactivity and increased pro-social behaviour. Additionally, there is a negative relationship between the ECERS-R scores on the space and furnishings subscale, and children's anti-social behaviours (Sammons et al., 2003b). Scores on the language and reasoning subscale are also positively related to children's cooperation and conformity skills (Sammons et al., 2003b). Further, Montes and colleagues (2005) have found considerable effect sizes linking high total score on ECERS-R with the reduction of existing socio-emotional risk factors and prevention of the emergence of new socio-emotional risk factors. A recent study, that used the Teacher-Child Rating Scale (T-CRS 2.1; Hightower et al., 1986) to measure children's social competence scores, has demonstrated a positive relationship between the Teaching and Interactions factor of the ECERS-R and children's assertiveness, frustration tolerance, task orientation, and peer social skills (Burchinal et al., 2008).

The issue of high-quality ECEC provision and teacher policy

High-quality ECEC provision is recognised by international literature, and state and federal governments as a pivotal policy target measure for the foundation of formal education. As stated earlier in this review, such targets are based on the individual (e.g., cognitive, language, and social development) and societal benefits (e.g., human capital development/accumulation) that are generated by high-quality ECEC provision. Hence, governments, especially those in the developed world, with a major responsibility for the funding of ECEC, have been allocating more resources and devoting greater policy attention to the quality of ECEC (International Labour Organization, 2012; New Zealand House of Representatives, 2013; OECD, 2013).

Over the past two decades, governments have been active in designing policies and programs that aim to improve the quality of ECEC and ensure access, irrespective of socio-economic status, to high-quality classroom environments (Dowling & O'Malley, 2009; Pianta et al., 2005). Often, these legislations focus on the manipulation of structural features such as child-to-staff ratio, class size, and staff educational attainment (Phillipsen et al., 1997). Studies have been conducted to identify influential characteristics that promote the quality of ECEC. For example, the relationship between teacher's level and type of education with the quality in ECEC has prompted policy makers and ECEC providers to consider requiring centres to increase the proportion of qualified registered teachers (Dowling & O'Malley, 2009; Elliott, 2006; Mitchell, 2010). The specific rationale behind any ECEC teacher policy (e.g., “bachelor's degree policy”) is that lead teachers with a bachelor's degree or higher in ECEC are expected to have higher-quality classrooms (Ackerman, 2005).

Teacher qualification as a predictor variable that influences the quality of ECEC

Teacher qualification has been identified by policy makers as one of the regulatable features and crucial variables that can predict quality in ECEC (Ackerman, 2005). It is, however, noted that the variable “teacher qualification” is not an intervention that was manipulated in the studies comprising the meta-analysis, but a characteristic of the teacher/care provider and a predictor variable in the studies.

Approved early childhood teaching qualifications vary between states and across countries. Based on current literature, teacher qualification can be categorised according to their level and type of education. Level of education can be referred to formal schooling in primary, secondary and tertiary systems. The tertiary system can be categorised as community college, junior college and university. The details of intervention may also include years of education, course credits in any specialized early childhood or child development-related education, and title of program. Further, differentiation of qualification is based on the type of program and the degree earned, such as associate degree (AA), Child Development Associate (CDA) Credential/AA/certificate in ECEC, bachelor's degree (BA), BA in ECEC or primary education, master's degree (MA), MA in ECEC or primary education, PhD, and EdD (professional doctorate). One should note, however, that some of the literature does not separate degrees of ECEC and that of other majors. MA and PhD are sometimes categorised as post-graduate education.

The extant literature

The extant literature, which includes primary studies and systematic reviews, has examined relationships between the common regulation or policy targets (e.g., the classroom features, staff characteristics, and administration) with the process quality of ECEC that is measured by standardised tools (e.g., ERS). Empirical evidence has been used to inform policy deliberation in this area (Vandell & Wolfe, 2000). Specifically, some large-scale studies of childcare quality were conducted, and researchers attempted to demonstrate relationships between program quality (as measured by the ECERS/ECERS-R/ITERS/ITERS-R) and variables such as staff stability and staff background characteristics (Huntsman, 2008; Phillipsen et al., 1997; Sakai et al., 2003).

Among the vast array of variables that claim to impact on the ECEC environment, high staff qualification has been identified as one of the strongest predictors of high ECERS/ITERS ratings (with a score of 5 or higher on a 7-point scale) (Burchinal, Cryer, Clifford, & Howes, 2002; Peisner-Feinberg, Burchinal, Howes, & Cryer, 1997; Phillipsen et al., 1997; Whitebook, 2003b; Whitebook, Sakai, Gerber, & Howes, 2001). By distinguishing bachelor's degrees and specialised child development-related education from other levels of education and training, previous literature suggests that bachelor's degrees with specialised training in ECEC secure high quality childcare and education outcomes (Ackerman, 2005; Kelley & Camilli, 2007; Whitebook, 2003a).

There are, however, some studies that suggest no significant relationship exists between ECERS scores and the percentage of teaching staff with a bachelor's or advanced training in ECEC in a centre. For example, Phillips, Mekos, Scarr, McCartney, and Abbott–Shim (2000) discover that teacher education did not significantly affect classroom processes for infants and children in preschool rooms. Also, focusing on the school-related learning and social skills over the pre-Kindergarten year, Howes and colleagues (2008) argue that there is a mixed relationship between teacher qualification and classroom quality (as measured by ECERS-R). Early et al. (2007) examined the results of seven major studies regarding the association between teacher qualification and ECEC quality and children's academic outcomes. On the whole they find no association, indicating that relying on policies that focus only on teacher qualifications is not enough to improve and enhance ECEC quality.

Whitebook (2003b) conducts a systematic review regarding the educational level of teachers and how the level of qualification potentially affects the early childhood learning environment. Whitebook's main findings suggest that teacher qualifications matter in terms of high-quality ECEC provision, and that ECERS ratings are positively affected by higher qualifications. To further Whitebook's review, and allow for more meaningful comparisons between individual studies regarding the impact of teachers’ qualification on the quality of the ECEC environment, we propose that the target population should be further disaggregated (e.g., sampling at the level of the classroom or at the level of the learning centre) and a common metric (e.g., effect size) for measuring outcomes be calculated.

A more recent meta-analysis was conducted by Kelley and Camilli (2007), who analyse the results of 32 studies (18 treatment-comparison studies and 14 studies with correlations between teacher education and outcomes). The authors examine the relationship between teacher qualifications and the ECEC environment. The authors aggregate four different constructs, including global classroom quality, teacher-child interactions, teacher pedagogical beliefs and knowledge, and classroom instructional activities, into a group of ECEC outcomes. Aggregating these constructs, however, makes it difficult to estimate the additional effect on a specific outcome (e.g., quality of learning environment) as a result of higher educational attainments by teachers (e.g., bachelor's degree or associate degree). In this review, we will not aggregate these constructs. Rather, we focus on quality in the early childhood environment and ratings in the ERS subscales to tease out the relationship between teacher qualifications and process quality in an ECEC setting. This specific information is critical for policy makers, as single measures inform many of their decisions.

Types of study designs

A preliminary exploration of published and unpublished literature focusing on authors who have contributed to understanding ECEC quality (e.g., Whitebook, Clifford, and Howes) was conducted to determine the period of time that should be covered in this review. Eligible studies, therefore, are those studies (both comparative and correlational) that examine the relationship between teacher qualification and quality of the ECEC environment (as measured by ECERS/ECERSR/ITERS/ITERS-R) from 1980 (this was when the ECERS was introduced) to 2014. To be eligible, a study must permit the identification of the education program (e.g., bachelor's degree) received by the lead teacher and provide a comparison between two or more groups of teachers with different educational qualifications. The details of the independent variable include year of education, course credits in ECEC and title of the program.

Since teachers’ education qualification cannot be controlled by the researcher, the study designs that will be included in the review are comparative and correlational non-randomized studies. In comparative studies, a group of teachers with high school education is compared to at least one group of teachers with other qualifications (i.e., AA, CDA Certificate, BA, BA in ECEC or primary education, MA, MA in ECEC or primary education, PhD and EdD). In correlational studies, analyses are typically conducted to explore the relationship between level of teacher education and study outcomes, reporting a correlation (e.g., via point biserial or Pearson correlation coefficient) between levels of education and ratings of classroom quality as measured by ERS.

Types of participants

The sample under consideration is ECEC settings where education programs are delivered to children by lead teachers and their assistants. In this review, ECEC programs may be delivered in indoor settings in either a centre-based or home-based classroom environment. We initially assumed that most studies would focus on centre-based classrooms, thus our primary population of interest is classrooms in ECEC centres that are affiliated with a state licensing agency. Specifically, participants in these ECEC settings are children from pre-kindergarteners and kindergarteners prior to elementary/primary school and centre-based providers.

Types of outcome measures

Eligible studies focus primarily on the quality of the early childhood learning environment as measured by the ECERS/ECERS-R/ITERS/ITERS-R. Seven categories of outcomes are identified according to the subscales and items of those tools, including, for example: Personal Care Routine; Space and Furnishings; Language-reasoning/Listening and Talking; Activities; Program Structure; Interaction; and Parents and Staff. Since some researchers may prefer using the two-factor scale (Activities/Materials and Language/Interactions) for follow-up analysis, these alternative outcomes are also included as applicable and possible.

Duration of follow-up

Multiple independent samples over time are not common for research in this area. Therefore, the literature is expected to be based on evaluation of outcomes in a single event. We did, however, find studies longitudinal in nature. That is, where a cohort is followed-up at subsequent time periods (e.g., one year after intervention, two years after intervention etc.) In such cases, we combined the longitudinal effects (i.e. correlation r) to produce an overall mean effect size (r) for this cohort. To do this, we produced a mean effect size (r) of the time points in those longitudinal studies. We did not treat effect sizes (r) of each time point as separate independent samples.

Types of settings

The review includes independent samples conducted in indoor centre-based settings that serve infants, toddlers, preschool and kindergarten children. Studies conducted in outdoor settings are not included in the review.

Selection of studies

We began our search for relevant studies by conducting a manual search of key journals for the period 1980 (this was when the ECERS was introduced) to 2014 – examples include, Early Childhood Research Quarterly, Early Childhood Research and Practice, Contemporary Issues in Early Childhood, Child Development, Applied Developmental Science, and the Journal of Child Psychology and Psychiatry. We then scanned relevant review articles, consulted the bibliographies of articles that meet the selection criteria and asked key researchers for assistance in identifying other relevant studies.

We searched a number of electronic databases covering the years 1980 to 2014. Compound terms (e.g., teacher qualification; staff education) were considered as a single term and entered into searches in quotes (i.e., “teacher qualification”). These databases include: Academic Search Premier; CBCA-Education; Cochrane Controlled Trial Register; Database of Abstracts of Reviews of Effectiveness (DARE); Dissertation Abstracts; EconLit; Education Full Text; Educational Resources Information Center (ERIC); Journal Storage Archive (JSTOR); Medline; Proquest Digital Dissertations; Proquest Direct; Project Muse; PsychInfo; Scopus; SocINDEX with Full Text; and SSRN eLibrary. Within each database, we searched using combinations of keywords from three categories:

1) Outcome: Scores/ratings OR Subscales OR Early Childhood Environment Rating Scale, ECERS/ECERS-R/Infant Toddler Environment Rating Scale, ITERS/ITERS-R

AND

2) Independent variable: Teacher's education OR Teacher's qualification OR Training OR Program OR Levels of qualification OR associate degree (AA) OR Child Development Associate (CDA) OR bachelor's degree (BA) OR BA in ECEC OR master's degree (MA) OR MA in ECEC OR PhD OR EdD

AND

3) Targeted population: Classrooms OR Centre-based (/Center-based) classrooms OR home-based classrooms OR Child care centres (/centers) OR Kindergartens OR Pre-schools OR Pre-K (/PreK) OR Pre-kindergartens (/Prekindergartens)

This strategy ensured that the database searched for the entire term rather than “teacher” AND “qualification”, which would clearly produce very different results. In addition, search terms with multiple iterations from a base word stem (e.g., quality) are typed in as word* (e.g., quality*). This approach enabled the researcher to capture relevant literature with fewer searches. Where a database included the function to manipulate the search field option, researchers limited the keyword search to title, abstract, reference list, whole document or a combination of fields. Results from a series of pilots indicated that the search ‘anywhere’ in the document option produced more hits with a lower inclusion percentage than searches conducted on the abstract only or title, abstract and descriptors. A fundamental objective in this review is to develop a search strategy that can be replicated. Consequently, the focus is to utilize electronic databases/resources that can be generally accessed (i.e., not restricted material through an organization's intranet).

Further, it is also important to locate grey literature or material that is not formally published (e.g., working papers, unpublished dissertations, and reports including government, non-government and technical reports etc.). Websites of any relevant government, research institutes and early childhood associations were searched for published and unpublished studies. Some relevant websites include, hfrp.org, melycaba.com, tnstarquality.org, sped.dpi.wi.gov, ceelo.org, ric.ed.gov, and aplus-education.co.uk.

Efforts were made to track down any unpublished studies highlighted in the search. Authors of published articles were also contacted to ascertain if additional results relating to our key outcomes are available. This strategy aimed to identify other studies that may be overlooked in previous reviews, also to identify relevant data that may have been omitted in the publication of these studies. In some cases, authors were able to provide more detailed information about their data (e.g., standard deviations) so that effect sizes can be calculated in a similar way.

Researchers recorded search information (date of search, database and search terms used), research information (design, method (s), agency, outcome, population) and reference information in a spreadsheet so that further interrogation of the data could be conducted at a future date.

Criteria for determination of independent findings

We are primarily interested in one outcome measure, process quality as measured by ECERS/ECERS-R/ITERS/ITERS-R. Therefore, the analysis is not affected by any bias that could result from data/outcome dependence. When a study provides data of the subscales, we extracted and coded the effect sizes for each subscale and conducted a separate analysis. Regarding the dependencies on the independent variables, multiple measures of teacher qualifications (e.g. credits of ECEC courses, years of education and level of education) have been used in some of the studies. For studies that report standardised mean differences and compare multiple treatment groups (e.g. teacher with Bachelor degree (as group 1), Associate degree (as group 2)) with a common control group (e.g. teachers with a High school diploma), we code these samples (e.g. Bachelor vs. High school (as sample 1) and Associate vs. High school (as sample 2) as dependent samples that require the use of a robust standard error model (see Section 3.2.4).

Details of study coding

Two of the authors of this review were responsible for reviewing and screening all titles and abstracts found through the search procedures. At the abstract screening stage, studies that were deemed as inappropriate would be those that do not involve the target participants (e.g., ECEC settings that serve all ages of pre-kindergarteners and kindergarteners prior to elementary/primary school) or are descriptive in nature where no relationship between teacher's qualification and ECEC quality is being measured.

The eligibility of relevant articles was screened based on the criteria for eligibility (see Section two “Criteria for Eligibility” in Appendix A). All eligible studies that meet the initial criteria were coded using an instrument developed by the authors to extract the specified information. The coding instrument includes items related to bibliographic information and source descriptors; analysis characteristics; sample characteristics; methodology/research design; and outcome data needed to calculate effect sizes.

All eligible studies were independently coded by two individuals to ensure the reliability of coding procedures and decisions. Coding schemes were refined to resolve any inter-rater discrepancies or differences. Discrepancies that remained unresolved were discussed and resolved by consensus with a third author.

The research team checked the references of eligible studies to determine if there were other studies of interest that were not retrieved in the original search. Any literature of interest was obtained and assessed for eligibility.

Statistical procedures and conventions

The correlation coefficient (r) was selected as the common effect size metric based on the frequency with which the different forms of data are present in the literature. We expected to locate a number of correlational studies, for which the appropriate effect size metric is the r. Where r's are used, the Fisher's r-to-z transformation is applied to all cases (Hedges & Olkin, 1985). We expected that for some studies (i.e., those with comparative designs), the scores on ERS and their subscales would be reported in natural units in the form of mean differences. When means and standard deviations were reported, we transformed these in to r using the method outlined in Lipsey and Wilson (2000). When means and standard deviations were not reported, effect sizes were calculated from other test statistics (e.g., t-tests, F-ratios) using formulas outlined by Lipsey and Wilson (2000). Where there were binary measures of the same outcomes, we used procedures recommended by Sanchez-Meca and colleagues (2003) to convert odds ratios (OR) to r.

Our selection of r as the most appropriate effect size for this study was because: (a) relative to alternative effect size metrics (e.g. Cohen's d and Hedge's g) r has a more direct relationship to other statistical concepts (e.g., statistical power and the general linear model); (b) it can be calculated across a wider range of study designs (correlation coefficients for dichotomous, ordinal, or continuous independent and dependent variables); and (c) it is base rate sensitive. That is, when base rates in a sample reflect those in the population, r provides a more realistic sense for how well one variable correlates to another (McGrath & Meyer, 2006).

We conducted 10 individual meta-analyses that examined the relationship between the level of teacher qualification and the overall ERS scores (main outcome), the seven subscales (e.g., Space and Furnishings, Activities, Interaction, Personal Care Routine, Parent and Staff, Program Structure and Space and Furnishing) and the two subscales (Language and Interactions and Provisions for Learning). Only analyses based on five or more observations were considered. Data synthesis was conducted using a specialized statistical meta-analysis software package (Comprehensive Meta-Analysis (CMA) version 2, 2006). For the calculation of meta-analytic mean effect size, the inverse variance weight (w) is calculated for each study using the appropriate standard error (Lipsey & Wilson, 2001). Regarding the method of variance estimate to compute the random effects mean, the method of moments using weighted least squares, which provides a closed solution for the random effects variance component (τ ²) (Raudenbush, 2009), is adopted by the CMA. The inverse variance was used so that effect sizes with greater precision (i.e., smaller variances) are weighted more heavily in the analysis. We employ a random effects model for pooling effects. An assumption is made that there are unexplained sources of heterogeneity across studies. The Q statistic, which was calculated in each fixed effect analysis, was used for the calculation of the τ ². In addition, we employ the I ² statistic (Higgins & Thompson, 2002) as an additional, albeit related, method of assessing heterogeneity for the fixed effect analysis. An I ² value greater than 50% suggests moderate heterogeneity even if the Q statistic is not statistically significant. All effect sizes are calculated using a 95% confidence interval. For the robust variance estimation approach, unrestricted, intercept-only meta-regression models were run in Stata using a macro provided by Tanner-Smith and Tipton (2013). The estimated mean effect sizes (correlation in this review) with robust standard error take into account dependences in the data due to the use of a same control group for multiple treatment group contrasts.

Forest plots have been used to display the results from the effect sizes. The plot displays the effect size, confidence intervals and significance level with respect to each study.

Moderator analysis

We examined possible moderators of process quality in ECEC settings. We began by including characteristics of the study participants and settings such as: (1) ECEC settings (e.g., centre-based vs. home-based classrooms); (2) baseline characteristics of teacher (e.g., age, gender, race/ethnicity); (3) country in which the study was conducted; (4) duration of follow-up (e.g., 1 year vs. 6 months); (5) outcome measure (e.g., ECERS, ITERS, ECERS-R and ITERS-R); and (6) dominant ethnicity of student group.

Where data was available, we analysed these potential moderating factors using an analog to the Analysis of Variance (ANOVA) and weighted least squares regression when appropriate. The analog to the ANOVA is a method of testing the ability of a single categorical value to explain excess variability in a distribution of effect sizes (Lipsey & Wilson, 2000). CMA was used to conduct moderator analyses. Forest plots are used to graphically display these moderator effects.

Missing data

In the event that a study did not provide suitable data for calculation of effect sizes (e.g., means and standard deviations, valid Ns), we endeavoured to contact the author(s) of the primary studies in order to obtain the missing information.

Sensitivity analysis

A sensitivity analysis was conducted to test the robustness of the conclusions with respect to the quality of the available data and approaches to the analysis. Any differences in terms of the results are examined according to our choice of effect size metric. We inspect funnel plots, and utilise ‘trim and fill’ methods that estimate correlation by adjusting for the number and outcomes of missing studies. The publication bias of any over - or underrepresentation of particular findings on the basis of their statistical significance were examined by analysing the funnel plot. When we plotted the observed effect sizes against their corresponding sample sizes a more or less symmetric (inverted) funnel is expected in the absence of this form of publication bias. An asymmetrical funnel plot indicates that studies with non-significant findings are suppressed from the published literature. The regression test for funnel plot asymmetry examines whether an association between the effect sizes and the corresponding sample sizes is present in the data. Validity and reliability of the findings would be challenged if such an association is present suggesting that a publication bias does exist (Sterne & Egger, 2005). Regression tests for funnel plot asymmetry were conducted for each of the eight meta-analyses.

Treatment of qualitative research

Qualitative studies was not included in the current study. We do, however, acknowledge seminal pieces of research in our discussion.

Results of the search

The systematic search identified 2023 unique sources (e.g., published or unpublished documents) on ECEC quality measured by ERS (i.e., ECERS, ECERS-R, ITERS, ITERS-R) and teacher qualifications. We were not able to obtain 12 sources to review for eligibility despite a number of different attempts (through our own efforts as well as through the employment of an information specialist). These sources tended to be university dissertations where the university and/or supervisors could not locate the author, and/or organizational reports. Several of these missing sources had ambiguous citations and may have been unlocatable because they were incorrectly documented in the online database. Of the 2011 sources we obtained, 130 studies were comparative studies and correlational studies, which reported on the relationship between teacher qualifications and ECEC quality, while 1867 of the sources did not report an independent sample on the relationship between teacher qualifications and ERS-measured ECEC quality and 14 of the sources were literature reviews, theoretical articles, or qualitative studies. Of the 130 quantitative studies reporting on a relationship between the two constructs (i.e., ECEC quality and teacher qualifications), 62 were excluded from the review because they had no comparative data and did not report on an outcome of interest. The remaining 68 studies that contained comparative information were further screened for suitability for meta-analysis. Of these, 20 studies did not provide enough data for the calculation of effect size (r). These were excluded. Two additional studies were identified as outliers and subsequently excluded from the analysis. The final set of 46 studies eligible for the meta-analysis contained 58 independent samples. All studies identified were subjected to scrutiny using the coding sheet (see Appendix 1). Table 1 displays the attrition of publications. Of the 46 included studies, 9 were theses (PhD =8; MA = 1). In the analysis of the overall ERS rating, 8 out of 44 were theses. In the analysis of the 7 factor ratings, 3 out of 14 were theses. In the analysis of the 2 factor subscales ratings, 1 out of 10 was a thesis.

Included studies

The 48 eligible studies differed according to their classification of teacher qualifications and type of studies, target population and use of measurement tool and outcome. The following section describes these differences, which are summarized in Table 2.

Sensitivity analyses

To examine the impact of our analysis decisions on the results, we conducted a series of sensitivity analyses. We used a funnel plot to inspect possible sources of bias in the data, including publication bias and small-study effects. We used a “trim and fill” method (Duval & Tweedie, 2004) to test for publication bias. We tested for small-study effects using an approach proposed by Egger (1997).

Overall ECEC quality

Figure 1 presents the funnel plot for the main outcome (overall quality) from the 44 eligible studies. Each point on the figure represents one outcome for one independent sample. Studies with a larger sample size have a smaller standard error, and such points are situated closer to the top of the funnel. Smaller studies are located toward the bottom of the funnel. We expect that large studies with positive and negative effects will be published, so there should be symmetry across both sides of the funnel at the top of the graph. However, in the presence of publication or small-study bias, we expect that the points appearing at the bottom of the funnel will be clustered on the positive effect side of the graph.

OPEN IN VIEWER

Figure 1:

Funnel plot of standard error by Fisher's Z: Overall ECEC quality measured by ERS

Figure 1 shows a fairly symmetrical distribution with no extreme results. A diagnostic test (Egger's test run in CMA as demonstrated in Rothstein, Sutton, & Borenstein (2005)) was conducted for further examination of potential publication or small-study bias. The above result demonstrates no evidence of publication or small-study bias, t (56) = 0.468, p = 0.641 (2-tailed).

Subscale level ECEC quality (7-factor classification)

Figure 2 presents the funnel plot for the seven subscale outcomes from the 14 eligible studies. Each point on the figure represents one outcome for one independent sample.

OPEN IN VIEWER

Figure 2:

Funnel plot of standard error by correlation coefficient: Seven-factor subscale ECEC quality measured by ERS

Figure 2 shows a somewhat asymmetrical distribution with a number of extreme positive results on the right. A diagnostic test (Egger's test run in CMA as demonstrated in Rothstein, Sutton, & Borenstein (2005)) was conducted for further examination of potential publication or small-study bias. The result demonstrated that there was no evidence of publication or small-study bias, t(86) = 0.658, p = .513 (2-tailed).

Subscale level ECEC quality (2-factor classification)

Figure 3 presents the funnel plot for the two subscale outcomes from the 10 eligible studies. Each point on the figure represents one outcome for one independent sample.

OPEN IN VIEWER

Figure 3:

Funnel plot of standard error by correlation coefficient: Two-factor subscale ECEC quality measured by ERS

Figure 3 shows a fairly symmetrical distribution with a number of extreme results on the left and the right. A diagnostic test (Egger's test run in CMA as demonstrated in Rothstein, Sutton, & Borenstein (2005)) was conducted for further examination of potential publication or small-study bias. The result demonstrated that there was no evidence of publication or small-study bias, t(32) = .334, p = .74 (2-tailed).

Overall ECEC quality

There were 44 eligible studies that measured overall ECEC quality as an outcome and provided a correlation for teacher or caregiver qualifications. Of the 58 eligible independent samples, 50 had a positive correlation (r greater than zero), indicating that for these independent samples higher teacher or caregiver qualifications were positively correlated with an overall ERS rating. Of the 50 independent samples with r greater than zero, 26 were statistically significant (p<.05).

Overall, the education level of the teachers or caregivers is positively correlated (small effect) to overall ECEC qualities measured by ERS. The estimated mean correlation (r) with robust standard error, assuming ρ = .80, was 0.198 (confidence limits 0.133, 0.263) (see Figure 4). The interpretation of correlation produced are: <.10 = trivial; .10 to .30 = small to medium; .30 to .50 = medium to large; and >.50 = large to very large (Cohen, 1988).

OPEN IN VIEWER

Figure 4:

Forest plot of correlation coefficients: Overall ECEC quality measured by ERS

According to Hedges and Hedberg (2007), educational researchers have indicated that effect sizes of 0.20 or smaller are of policy interest when they are based on measures of academic achievement or quality of education program. This result also highlights the practical significance of having ECEC program providers with higher education as lower effect sizes are usually attained on assessment of environmental settings based on ERS.

The differences between studies contributed significantly to the variation in effect sizes. The 58 independent samples were significantly heterogeneous according to the Q statistic (Q (57) = 143.85, p < .001). The large I ² statistic indicated that 60% of the variance in the correlation coefficient may be a result of study factors (I ² = 60.377, τ² = .012, se = .005).

Moderator analyses conducted in CMA showed a significant variation in correlations between studies according to the ethnicity of the dominant students and the different ERS used (see Table 3). The moderator analysis of ethnicity produced indicates that, overall, ECEC quality is positively correlated with teachers’ qualification except for the Latino subgroup. The moderator analysis of scale used confirmed that the available standardised ERS measurements used across studies do not contribute to any significant difference about the relationship between teacher qualifications and ECEC quality. Some potential moderators such as socioeconomic status were not included due to the limited specification of such characteristics of the ECEC centres or target population among existing studies.

Subscale outcomes

Space and furnishings

Thirteen studies provided outcome data on space and furnishings, giving 14 independent correlations overall. Figure 5 summarizes the 14 independent samples included in the meta-analysis on space and furnishings (note that one study contained two samples). Eleven of the fourteen independent samples had a positive correlation (r greater than zero), indicating that for most included studies, higher teacher qualifications was correlated with higher ratings on space and furnishings. Of the 11 independent samples with a correlation greater than zero, five were statistically significant at the .05 level.

OPEN IN VIEWER

Figure 5:

Forest plot of correlation coefficient: Space and furnishings quality measured by ERS

Overall, the interventions were associated with a small and non-significant increase in rating on space and furnishing (p=.144). The overall r for the 14 independent samples combined was 0.122 using a random effects model, and the 95% confidence interval included zero (lower limit = -.042, upper limit = 0.280).

The 14 independent samples were significantly heterogeneous according to the Q statistic (Q (13) = 122.662, p =.000). The I ² statistic indicated that 89% of the variation in the correlation coefficient could be attributed to study-level factors (I ² = 89.402, τ²= .074, se = .002).

Activities

Thirteen studies measured rating on activities as an outcome, contributing a total of 14 effect sizes (r) to the meta-analysis (note that one study contained two independent samples). Of the 14 correlations, 12 had a positive r, indicating that for these studies higher teacher education level was correlated with higher ratings on activities. Of the 12 independent samples with a positive correlation coefficient, seven were statistically significant at the .05 level.

Despite some individual studies showing no correlation between teacher qualifications and activities, the meta-analysis showed an overall positive correlation. The weighted correlation coefficient for the 14 independent samples combined was .204 using a random effects model (see Figure 6). The 95% confidence interval for r did not include zero (lower limit = .140, upper limit = .267).

OPEN IN VIEWER

Figure 6:

Forest plot of correlation coefficient: Activities quality measured by ERS

The 14 independent samples were not significantly heterogeneous according to the Q statistic (Q (13) = 18.317, p = .146). The I ² statistic indicated that 29% of the variance in r may be a result of study factors (I ² = 29.030, τ² = .004, se = .006).

Interaction

Twelve studies measured rating on interaction as an outcome, contributing a total of 14 correlations to the meta-analysis (note that two studies contained two independent samples). Of the 14 effects, 11 had a positive r, indicating that for these studies higher teacher education level was correlated with higher ratings on interaction. Of the 11 independent samples with a positive r, 5 were statistically significant.

Despite some individual studies showing no correlation between teacher qualifications and interaction, the meta-analysis showed an overall positive correlation. The weighted mean r for the 14 independent samples combined was .122 using a random effects model (see Figure 7). The 95% confidence interval for r did not include zero (lower limit = .053, upper limit = .189). The 14 independent samples were not significantly heterogeneous according to the Q statistic (Q (13) = 19.796, p = .100). The I ² statistic indicated that 34% of the variance in r may be a result of study factors (I ² = 34.331, τ² = .005, se = .006).

OPEN IN VIEWER

Figure 7:

Forest plot of correlation coefficient: Interaction quality measured by ERS

Language and reasoning

Twelve studies measured rating on language and reasoning as an outcome, contributing a total of 14 effect sizes to the meta-analysis (note that two studies contained two independent samples). All of the 14 independent samples had a positive r, indicating that for these studies higher teacher education level was correlated with higher ratings on language and reasoning. Of the 14 independent samples with a positive r, five were statistically significant.

The meta-analysis showed an overall positive correlation between teacher qualifications and ratings on language and reasoning. The weighted mean r for the 14 independent samples combined was .203 using a random effects model (see Figure 8). The 95% confidence interval for r did not include zero (lower limit = .122, upper limit = .282). The 14 independent samples were significantly heterogeneous according to the Q statistic (Q (13) = 26.049, p = .017). The I ² statistic indicated that 50% of the variance in r may be a result of study factors (I ² = 50.094, τ² = .011, se = .009).

OPEN IN VIEWER

Figure 8:

Forest plot of correlation coefficient: Language and reasoning quality measured by ERS

Parents and staff

Seven studies measured rating on parents and staff as an outcome, contributing a total of eight effect sizes to the meta-analysis (note that one study contained two independent samples). Of the eight independent samples, seven of them had an r greater than zero, indicating that for these studies higher teacher education level was correlated with higher ratings on parents and staff. Of the seven independent samples with an r value greater than zero, three were statistically significant.

The meta-analysis showed an overall positive correlation between teacher qualifications and ratings on parents and staff. The weighted mean r for the eight independent samples combined was .189 using a random effects model (see Figure 9). The 95% confidence interval for r did not include zero (lower limit = .049, upper limit = .321). The eight independent samples were significantly heterogeneous according to the Q statistic (Q (7) = 21.219, p = .003). The I ² statistic indicated that 67% of the variance in r may be a result of study factors (I ² = 67.011, τ² = .023, se = .021).

OPEN IN VIEWER

Figure 9:

Forest plot of correlation coefficient: Parents and staff quality measured by ERS

Personal care routines

Eleven studies measured rating on personal care as an outcome, contributing a total of 12 effect sizes to the meta-analysis (note that one study contained two independent samples). Of the 12 independent samples, six had a positive r, indicating that for these studies higher teacher education level was correlated with higher ratings on personal care. Of the six independent samples with a positive r, three were statistically significant.

The meta-analysis showed an overall positive correlation between teacher qualifications and ratings on personal care routines. The weighted mean r for the 12 independent samples combined was .095 using a random effects model (see Figure 10). The 95% confidence interval for r included zero (lower limit = -.053, upper limit = .239).

OPEN IN VIEWER

Figure 10:

Forest plot of correlation coefficient: Personal care quality measured by ERS

The eight independent samples were significantly heterogeneous according to the Q statistic (Q (11) = 69.022, p = .000). The I ² statistic indicated that 84% of the variance in r may be a result of study factors (I ² = 84.063, τ² = .050, se = .030).

Program structure

Eleven studies measured rating on program structure as an outcome, contributing a total of 12 effect sizes to the meta-analysis (note that one study contained two independent samples). Of the 12 independent samples, eight had a positive r, indicating that for these studies higher teacher education level was correlated with higher ratings on program structure. Of the eight independent samples with a positive r, three were statistically significant.

The meta-analysis showed an overall positive correlation between teacher qualifications and ratings on program structure. The weighted mean r for the 12 independent samples combined was .224 using a random effects model (see Figure 11). The 95% confidence interval for r did not include zero (lower limit = .014, upper limit = .415).

OPEN IN VIEWER

Figure 11:

Forest plot of correlation coefficient: Program structure quality measured by ERS

The eight independent samples were significantly heterogeneous according to the Q statistic (Q (11) = 141.493, p = .000). The I ² statistic indicated that 92% of the variance in r may be a result of study factors (I ² = 92.226, τ² = .119, se = .069).

Two-factor classification: Language and interaction

Nine studies measured rating on language and interaction as an outcome, contributing a total of 17 effect sizes to the meta-analysis. Of the 17 independent samples, 11 had a positive r, indicating that for these studies higher teacher education level was correlated with higher ratings on language and interaction. Of the 11 independent samples with a positive r, nine were statistically significant. The meta-analysis showed an overall positive correlation between teacher qualifications and ratings on language and interaction. The estimated mean correlation (r) with robust standard error, assuming ρ = .80, was 0.096 (confidence limits -0.172, 0.363) (see Figure 12).

OPEN IN VIEWER

Figure 12:

Forest plot of correlation coefficient: Language and interaction quality measured by ERS

The nine studies were significantly heterogeneous according to the Q statistic (Q (16) = 583.703, p =.000). The I ² statistic indicated that 97% of the variance in r may be a result of study factors (I² = 97.259, τ² = .206, se = .0122). We noted that there were some erratic patterns associated with the Vu et al (2008) study (see Figures 12 and 13). One possible explanation for the variance in the mean difference between the various sub-groups may be differences in sample characteristics. In particular, ERS ratings in school district and state preschool are negatively correlated with teacher qualification. This is particularly unusual, as we would expect the relationship between the two variables to be consistent across samples within the study.

OPEN IN VIEWER

Figure :

Forest plot of correlation coefficient: Provisions for learning quality measured by ERS

Statistical conclusion validity

Statistical conclusion validity (or statistical validity) is concerned with issues or problems that threaten valid inference-making (Kirk, 1995). According to Elvik (1999), results of meta-analyses cannot be theoretically valid without being statistically valid, at least with respect to some of the criteria of statistical validity. The statistical validity attained in a meta-analysis depends on a list of criteria including choice of techniques of analysis, commensurability of dependent variables, evidence of publication bias, and shape of distribution.

Based on our diagnostic analysis, we adopted a random effects model for some of the reported results due to the identified random heterogeneity of the research subjects. Also, whenever studies with dependent samples were present, we adopted the robust standard error model for the analysis. With regards to the dependent variable of the included studies, the dependent variable is the ratings of measures of ECEC quality using comparable ERSs (i.e. ECERS, ITERS and their revised version). In this respect, our set of included studies with commensurable definitions of dependent variables can be regarded as more valid from a purely statistical point of view than a set of studies with incommensurable definitions of dependent variables. Regarding to potentials of publication bias, several diagnostic tests (e.g. Egger's test, Duval and Tweedie's “trim and fill” method) were performed to demonstrate that there was no evidence of publication or small-study bias. In all, we feel confident that our systematic search captured a large number of unpublished and small studies, since a substantial proportion of our eligible studies are dissertations and unpublished reports. Therefore, the effect of publication bias introduced across the studies is likely to be trivial. Concerning the shape of the distribution of results in our set of studies, the funnel graph diagram for ratings of overall ECEC quality presents a fairly symmetrical distribution with no extreme results, while the diagrams for sub-factor ratings present a somewhat asymmetrical distribution with a number of extreme positive results on the right.

Internal validity

Internal validity focuses on the quality of study design examining the influence of confounding variables. For example, the potential effect or contribution of any uncontrolled/measured variables on the evaluated relationships among concerned variables. A randomized experiment is commonly recognized as the most preferable design that has the highest possible internal validity as it avoids all the potential threats (e.g. preexisting differences between experimental and control conditions) (Farrington, 2003). The UK Centre for Reviews and Dissemination (2001) proposed a hierarchy of research designs with five types of evidence: (1) randomized, controlled, double-blind trials; (2) quasi-experimental studies (experiments without randomization); (3) controlled observational studies (comparison of outcomes between participants who have received an intervention and those who have not); (4) observational studies without a control group; and (5) expert opinion.

In the context of this review, both the randomized experiment trail and quasi-experimental study designs were not adopted as there was no evidence that teachers with different qualification were randomly allocated to an ECEC classroom/center. In addition, a pre-post test experiment was not adopted as most of the studies were cross-sectional controlled observational studies. The class and classroom being evaluated did not have the same chance of being assigned to be managed by a lead teacher with a specific qualification. Further, it was possible that the comparison group differences with regards to ECEC qualities could be attributed to both observed and unobserved characteristics due to systematic cofounding factors (e.g. the classroom characteristics). For example, ECEC centers located in districts with a lower socio-economic level might have a relatively lower overall rating on ECEC quality when compared to the ratings of centers located in districts with a higher socio-economic level. A lower-income ECEC center may not have the finance to support the management of its classrooms or hire more well-educated/qualified teachers. Some of the nation-wide and multi-state/large-scale studies in this review might have aggregated and compared classroom ratings without controlling for the potential influence of classroom-/center-specific characteristics. However, it is noted that these potential issues do not apply to studies that compared the ECEC quality of multiple classrooms within the same center.

Construct validity

Studies in our review are generally expected to have a reliable construct validity with respect to the accuracy of the operational definition and measurement of the theoretical constructs (ECEC qualities) which guide the evaluations. The use of ERS and its validity has been tested in the ECEC sector.

External validity

Regarding the ability to generalize the results of our current review to other subjects and settings, our moderator analyses suggest that the evidence do not differ across classrooms with different ethnicity or classrooms that were being measured by different ERS.

Importance for practice and policy

The results from this study provide an up-to-date analysis of the correlation between teacher qualifications and the quality of the early childhood learning environment. Based on published research studies since the 1980s, the review provides an in-depth understanding relevant for current practice and policy. The review was able to draw upon information from a number of countries, demonstrating that the relationship between teacher qualification and enhanced quality of the learning environment was not dependent on context or culture.

Results from this study are important for policy makers wanting to enhance policy and practice within ECEC settings to improve children and family outcomes. This meta-analysis provides evidence of a significant and positive correlation between teacher qualification and quality with respect to the learning environment for all young children in ECEC settings, including infants and toddlers. Mandating qualified teachers (with tertiary education) may lead to significant improvement for both process and structural quality within centre-based and home-based ECEC settings.

The economic investment in qualified staff with some tertiary education to enhance the quality of the learning environment may also have lasting affects for children, families and society. Research has shown that children in high quality ECEC settings grow up to be healthier and more productive adults. Government investment in ECEC is internationally recognized as a sound economic driver for the well-being of a nation (Britto, Engle & Super, 2013). This means that improvements in quality in ECEC settings leads to enhanced educational, societal and economic benefits.

Future research

This meta-analysis provides strong evidence for the relationship between teacher qualification and classroom quality as measured by the ERS. It has been able to provide an up-to-date analysis based on research since the 1980s. While this study has focused on quantitative research outcomes, further research in this area may also want to include an analysis of qualitative data to explore the relationship between teacher qualification and the quality of early childhood learning environment. In addition, future studies may seek to establish a causal relationship between teacher qualification and classroom quality using natural experiments or non-experimental designs. Such analysis would complement current research and provide a more in-depth understanding of the early childhood learning environment.

Another key consideration for further research would be the consideration of moderators such as socioeconomic status of the ECEC settings or target population among existing studies. Given the limited specification of such characteristics, this review was unable to use this potential moderator.

Further research is also needed into the specific knowledge and skills that are learnt by teachers with higher qualifications that enable them to complete their roles effectively. It would be important for research to identify key enablers during professional training that provide teachers with confidence and competence to complete their role as a teacher effectively. By identifying such enablers, training can become better geared to the learning and development of qualified teachers.