Abstract
A tenet of multitiered systems of support and response to intervention (MTSS-RTI) is that lack of response to instructional intervention is explained by classroom experiences and behaviors given opportunities to learn. We investigated the potential of filling this information gap in MTSS-RTI decision making using ecobehavioral observation to inform steps that could be taken for children not responding to preschool literacy instruction. Data analyses indicated that (a) teachers implemented a uniform pattern of daily activities providing children with infrequent opportunity to learn literacy, (b) the proportion of children’s co-occurring academic engagement also was low but varied widely depending on the activity and teacher’s literacy focus, and (c) children’s personal risk characteristics moderated the strength of relationships. Novel was the finding that in some activities and teacher behaviors, teachers appeared to be differentiating instruction benefiting children with individualized education programs and dual-language-learner risk. Implications are discussed.
Keywords
It is challenging for preschool teachers to meet the needs of all the children they serve because of the diversity in the language and early literacy skills children bring to the classroom. Reports indicate that preschool children in the lower quartile of language and literacy skills proficiency are often those with limited prior experience, low socioeconomic status (SES), dual-language experiences, or developmental delays (Carta & Driscoll, 2013; Neuman, Kaefer, & Pinkham, 2018). Children in the lower quartile make progress in a preschool year but are not likely to catch up to typically developing peers (Greenwood et al., 2013). These children remain at risk for not being ready for school and not learning to read proficiently (Hoff, 2006, 2013; Zill & Resnick, 2006). It has been suggested that response to instructional intervention predicts which children readily learn literacy skills and why others struggle (Connor et al., 2009; Connor, Morrison, & Petrella, 2004). The approach known as multitiered systems of support and response to intervention (MTSS-RTI; Division for Early Childhood of the Council for Exceptional Children, National Association for the Education of Young Children, & National Head Start Association, 2014; VanDerHeyden & Snyder, 2006) uniquely addresses this diversity in preschool children’s entry skills by letting RTI data inform decisions about the need for more or less intensive (differentiated) instruction (Merrell & Buchanan, 2006).
In MTSS-RTI, universal screening and progress-monitoring data provide the evidence that practitioners need when making intervention decisions (Batsche, Castillo, Dixon, & Forde, 2008; Snyder, Wixson, Talapatra, & Roach, 2008). Significant progress has been made in developing the approach for preschool (Carta et al., 2016). For example, Individual Growth and Development Indicators (IGDIs) for infants, toddlers, and preschoolers provide early educators with the MTSS-RTI measurement they need (Greenwood, Carta, & McConnell, 2011; McConnell, Wackerle-Hollman, & Bradfield, 2014; Neuman & Carta, 2011).
Although IGDIs are good at indicating who may benefit from more intensive instruction (McConnell et al., 2014), they provide little help in determining what may be contributing to an individual child’s lack of learning progress (Why is the learning problem happening?), what steps should be taken going forward (What should be done about the problem?), and whether these steps are being implemented as planned (Is a solution being implemented?). This is illustrated in Tilly’s (2008) problem-solving model (Figure 1). We know of no methods that inform the steps to be taken after problem identification, particularly ones that bring greater precision to these questions.
Tilly’s multitiered-system-of-support universal screening and progress-monitoring model (Tilly, 2008).
Ideally, a useful method would estimate a focal child’s opportunity to learn language and literacy as well as the child’s behavior co-occurring with these opportunities to learn. When considered together, they indicate the success or failure of instruction in providing the necessary experiences needed to promote literacy. Both variables could be altered by the teacher either by providing greater exposure to target content or by embedding use of evidence-based practices known to promote child responding and academic engagement. These variables could be operationalized as the (a) proportion of observed time that learning opportunities are provided by the teacher and (b) the likelihood of a child’s engaged response occurring given the instructional opportunity, an instruction–child behavior dependency. This method would enable early childhood practitioners to collect and interpret these indicators for individual children and to make data-based decisions about implementing changes using evidence-based practices (e.g., Ziolkowski & Goldstein, 2008) with implementation supports (e.g., practice-based coaching; Snyder, Hemmeter, & Fox, 2015) that could be tested for function.
Widely used preschool observational measures currently do not provide this information. They are not designed to inform decisions about what children should be taught and what they should learn in preschool (Shanahan & Lonigan, 2008), do not measure child-level information (CLASS; Pianta, La Paro, & Hamre, 2008), or do not measure instruction–child behavior co-occurrence (Downer, Booren, Lima, Luckner, & Pianta, 2010). Those few that do measure child-level information and child behavior co-occurrences require video capture and coding software to be used outside of the classroom and are not yet feasible for practitioners (Connor et al., 2009; Pelatti, Piasta, Justice, & O’Connell, 2014).
Ecobehavioral observation data (i.e., the Code for Interactive Recording of Children’s Learning Environments [CIRCLE]) are a potential solution to all of these problems (Greenwood et al., 2018) because they provide the needed information on a child’s opportunity to learn and instruction–child behavior co-occurrence (Kontos & Keyes, 1999; D. Powell, Burchinal, File, & Kontos, 2008). CIRCLE is the outgrowth of ecobehavioral observational research (e.g., Greenwood, Carta, Kamps, Terry, & Delquadri, 1994) based on the premise that child behavior is malleable and responsive to situational variations provided by teachers in the classroom environment (Fredricks, Blumenfeld, & Paris, 2004).
Guiding CIRCLE development was the concept that a child’s learning from classroom instruction is a function of the interaction between the opportunity to learn, child behavior given opportunities, and child risk characteristics (i.e., low SES, individualized education program [IEP], or dual-language learner [DLL]). CIRCLE is designed to inform these considerations in intervention decision making. Regarding the opportunity to learn, we analyze the composite occurrence levels of teacher literacy focus (TLF; What is taught?) and the activity structures (Where is it taught?). CIRCLE’s TLF composite, for example, reflects the opportunity to learn phonological awareness, alphabet and print concepts, story comprehension, other comprehension, vocabulary, reading, and literacy involvement provided by the teacher (see Table 1). Preschool teachers organize the day in terms of activities in which literacy content can be embedded in contrast to the subject matter content that drives elementary education schedules. CIRCLE activity structures include center, story time, large and small groups, and individual activities (consolidated as “academic activities”) and meals and snacks; cleanup, setup, and transition; personal care; therapy; restricted access; and none listed (consolidated as “other, nonacademic activities”) (Table 1).
CIRCLE Descriptives.
Note. CIRCLE = Code for Interactive Recording of Children’s Learning Environments.
For questions concerning children’s behavior (What is the child doing?), we consolidate CIRCLE’s 13 child behaviors into three theoretical composites (Greenwood et al., 2018). These are children’s academic engagement (CAE), other engagement, and other behavior (see Table 1). CAE is the sum of writing, reading words or letters out loud, academic manipulation, academic verbal response, and academic attention. Compared to prior reports that excluded academic attention in CAE (Greenwood, Horton, & Utley, 2002), we included academic attention here because preschool children’s repertoire of classroom academic skills is only just emerging (see Table 1). Other engagement is the sum of play, singing and music, nonacademic manipulation, gross motor, eating and drinking, and nonacademic attention. Other behavior is a composite of inappropriate behaviors (e.g., aggression, noncompliance) and any other child behaviors not otherwise defined in CIRCLE.
For questions about the likelihood of a positive relationship occurring between instruction and child behavior, we analyze the data preserving co-occurrence as the focal variable (Kontos, Burchinal, Howes, Wisseh, & Galinsky, 2002). Because CIRCLE data are multiply time sampled, the probability of a child’s behavior B, given TLF A (an instruction–child behavior dependency), can be estimated. These data help confirm the extent to which academic engagement is occurring at the moments in which the child actually has the opportunity to learn and uniquely reveals the activities wherein desired child behaviors are occurring differentially. This information is particularly useful in analyzing RTI and differentiating instructional interventions (D. Powell et al., 2008) because it provides a window on the opportunity to learn and exactly how a focal child is responding to a given intervention. With this information, teachers can do more or less of what is working or add strategies to those not working to boost child response when lacking or inappropriate.
Preliminary findings have supported CIRCLE’s potential based on analyses of an extant CIRCLE data sample (Greenwood et al., 2018). That extant sample was part of data collected in a 2-year wait-list randomized control trial of the efficacy of an enhanced literacy intervention (Greenwood, Abbott, Beecher, Atwater, & Petersen, 2017). Thirty-nine lead teachers (20 in Year 1, 19 in Year 2) participated. The child sample with CIRCLE observations included 117 (59 in Year 1, 58 in Year 2) who were all 4 to 5 years old. Twenty-seven percent of children had an IEP in Year 1, versus 28% in Year 2. None were DLLs.
Key findings indicated that children in most classrooms prior to intervention experienced low levels of exposure to literacy-focused instruction (Greenwood et al., 2018). We also reported wide variation in the amount of the literacy-focused instruction teachers embedded during daily activities. For example, children were much more likely to experience literacy instruction during story time and large groups, and least likely to experience literacy instruction during centers, small groups, and other activities. Other researchers have reported that limited opportunity maybe a reason for some literacy learning problems and that improvements could result by increasing exposures to some activities over others. For example, teachers could reduce time in centers and other activities in favor of spending more time in large groups and story time. Teachers could also embed more literacy-focused instruction during these times (Downer, Rimm-Kaufman, & Pianta, 2017; Vitiello, Booren, Downer, & Williford, 2012).
CAE also has been positively associated with language and literacy outcomes and readiness (Chien et al., 2010; Sabol, Bohlmann, & Downer, 2018); however, the activities preschool teachers provide are highly variable in promotion of CAE. D. Powell, Burchinal, File and Kontos (2008) reported that children were involved in whole-group settings for 52% of the observation time during which they were least likely to be actively engaged. Children were more likely engaged during academic activities when involved in a peer group and given teachers’ affirmations or monitoring. We also reported that the proportion of CAE was low, only .30 overall, but much more likely given the teacher provided a focus on literacy (.68) compared to nonliteracy (.32) (Greenwood et al., 2018).
Results also shed light on the challenges teachers face given the finding that children’s typical instruction–response dependencies were moderated by children’s literacy risk and IEP status. Children at greater risk due to weaker literacy skills or special needs (an IEP) had significantly weaker instruction–response dependencies compared to children without these risks in some but not all situations. For example, children at greater literacy risk and children with an IEP had weaker story time–CAE dependencies compared to children with low literacy risk and without IEPs. These results appeared to be helpful information in guiding how instructional supports may need to be differentiated (intensified) to strengthen the likelihood of CAE for higher-risk children in specific activities.
Our general thesis is that the addition of CIRCLE observation data to preschool MTSS-RTI decision making will overcome present limitations in logic, precision, and evidence. For example, CIRCLE observation data conducted by an MTSS-RTI team member (consultant or coach) could be used to investigate, analyze, and report information that informs the questions, Why is the problem happening? What should be done about it? Is the solution being implemented? and Is it working? (see Figure 1). The long-term goal is to improve understanding of the effects of preschool language and literacy instruction through descriptive and nomothetic research, and evidence from ideographic modifications of children’s interventions.
The specific purpose of this research, however, was to replicate CIRCLE’s sensitivity to variations in classroom instruction and children’s co-occurring academic engagement as reported previously. Replications are important because they test new research questions, apply new statistical methods, and check the reproducibility of earlier findings given that replication efforts often produce no effects or weaker effects than originally reported (Open Science Consortium, 2015). In addition to replicating Greenwood et al. (2018), we were able to examine the moderating effects of DLL status. We addressed the following questions.
To what extent did teachers provide language- and literacy-focused instruction (opportunities to learn) in daily activities? This question sought to replicate earlier findings regarding infrequent literacy opportunity levels provided children during daily activities and overall, when the opportunity to learn is one link to understanding a child’s lack of response to instruction.
Was children’s academic engagement associated with momentary variations in activities and literacy-focused instruction? This question sought to replicate earlier findings regarding whether the probability of desired children’s academic behavior varied significantly given activities and language and literacy-focused instruction—another link to understanding a child’s lack of response to instruction.
Do children’s personal risk characteristics moderate instruction–child behavior dependencies? This question sought to replicate earlier findings regarding whether personal risk characteristics (i.e., literacy risk and IEP status) strengthened or weakened children’s instruction–response dependencies in various situations and also extended this analysis to include children with DLL status not previously examined. The work is important because it informs the potentially negative function of risk factors on children’s response to instruction but also may confirm the presence of instructional differentiation in the case of children at risk.
Method
Background
Both the initial report (Greenwood et al., 2018) and the current investigation used extant data samples in secondary analyses. The sample used in this replication study was first reported by Greenwood et al. (2013). That study was a longitudinal descriptive comparison of the progress children in made in one preschool year. Language and literacy outcome measures were both formative and summative. CIRCLE proportion occurrence data were used as an indicator of CAE and TLF. Primary findings indicated that (a) all children in their preschool year prior to kindergarten made small academic gains, but children starting the year in lower Tiers 2 and 3 performance levels did not close skills gaps; and (b) variations were noted by program types of varying sociodemographics (i.e., pre-K, Head Start, Title I, and tuition based) and in CIRCLE TLF exposure and CAE occurrence.
Although the two extant samples were common in terms of measures used and 4- and 5-year-old participants, there were differences. For example, the Greenwood et al. (2013) sample was collected 4 years earlier than Greenwood et al. (2017). Sample also varied in numbers of sites (N = 4 vs. 2), programs (N = 65 vs. 2), and children enrolled (N = 659 vs. 117), respectively. CIRCLE data were collected on one occasion for each child at midyear in Greenwood et al. (2013) versus multiple occasions gathered sequentially over 2 years (Greenwood et al., 2017). The 2013 sample included pre-K, Title I, Head Start, and tuition-based programs; 9% of children had IEPs and 23% were DLLs. The 2017 sample included two programs (one reverse inclusion vs. one state-funded pre-K); 15% had IEPs and there were no DLLs.
Sample
Programs, children, and teachers
The children reported by Greenwood et al. (2013) were recruited from programs with language and literacy goals and that were using an early literacy core curriculum with an identifiable scope and sequence; the majority of early literacy instruction occurred in English. For the current analysis, a subsample of 354 children with complete observation and literacy screening data participated. This subsample was stratified to represent classroom differences in children’s literacy skills by randomly drawing six per classroom to receive observations (see Table 2). For various reasons of missing data and efforts to replace data when children moved away early in the study, the number of children who were actually assessed in each classroom ranged from two to seven (see Table 2). The percentages of children in the three literacy risk groups were 28%, 46%, and 26% for the high-, mid-, and low-risk groups, respectively (see group cut points below); 9% had IEPs; and 24% were DLLs. As a group, DLLs were more precisely defined as (a) Spanish-English DLLs but most comfortable speaking Spanish (15%), (b) Spanish-English DLLs but most comfortable speaking English or English and Spanish (5%), and (c) DLLs speaking languages other than Spanish (4%), for example, Hmong. All enrolled teachers and staff provided informed consent. Classroom lead teachers reported an average of 10.4 years of teaching experience, and only 5% were nondegreed. The reported degrees in early childhood were graduate (20%), 4 year (45%), 2 year (8%), and Child Development Associate (3%). Nineteen percent had degrees in other fields.
Literacy Skills Sample Characteristics.
Note. N = 354 children, 65 classrooms. IEP = individualized education program; DLL = dual-language learner.
F(63, 286) = 2.3332, p = .0001.
F(1, 338) = 1602.2, p = .0001.
F(1, 338) = 3.730, p = .05.
Measurement
Child and family characteristics
Sociodemographic characteristics of the children and their families were assessed using a 25-item parent survey. Date of birth, age, gender, race-ethnicity, and disability status indicated by the IEP were collected for each child, along with the primary caregiver’s educational attainment and languages spoken at home (DLL status).
Children’s preliteracy screen
The Get Ready to Read (GRTR; Whitehurst & Lonigan, 2001b) was used (Phillips, Lonigan, & Wyatt, 2009; Whitehurst & Lonigan, 2001a, 2001b). This screener is a brief, 20-item multiple-choice measure of letter-names and letter-sounds, print knowledge, emergent writing, and phonological awareness with alpha = .78, test-retest reliability = .80, and criterion validity = .58 to .69 (Phillips et al., 2009). The raw score was used in analyses.
CIRCLE measurement
The CIRCLE data reported originally (Greenwood et al., 2013) were used to describe and quantify (a) classroom activities (e.g., story, small groups), (b) teacher’s behavior (e.g., focus of instruction), and (c) a target child’s behavior (i.e.., academic engagement, other engagement) using a 15-s, momentary time sample method (Kennedy, 2005). Momentary time sampling is the best sampling approach for estimating frequency of occurrence (Ary, 1984; J. Powell, Martindale, & Kulp, 1975). CIRCLE data were recorded on tablet devices using a data entry application (Atwater, Lee, Montagna, Reynolds, & Tapia, 2009) that paced data entry by timing the intervals.
In momentary time sampling, observers record events simultaneously when prompted by an unobtrusive signal. At the signal, observers selected the event best describing the activity occurring at that moment from a drop-down entry field in the application. Teacher’s behavior and child events were recorded in a similar manner in the next two intervals. The activity, teacher, and child recording sequence was repeated until the end of the observation. Observations of the teacher lasted 90 min, during which time three children were sequentially observed for 30 min each. Children’s observations rotated through 10-min blocks to represent variability in the activities children were experiencing. The remaining three children were observed the next day. Observations were scheduled by the teacher’s report of when literacy-related activities were most likely to occur (e.g., centers, story time).
A single measurement director trained, planned, and supervised the multisite data collection. This director worked with three cross-site coordinators who supervised and monitored implementation. Local coordinators were trained and certified by the CIRCLE’s developer. All staff members at sites were trained to use the same measures prior to study data collection. All met calibration standards on procedural and measurement reliability, and reliability checks were conducted using 20% of observations at each site. The reliability estimates for CIRCLE data were 84.6% to 97.5% for exact percentage agreement overall; kappa ranged from 0.70 to .88 per study site on individual CIRCLE codes (Greenwood et al., 2013).
Statistical Analysis
As with Greenwood et al. (2018), sample representativeness and distributional features of the data were examined using simple descriptive analyses (i.e., frequencies, means and standard deviations, proportions, and graphical displays). Like Greenwood et al. (2018), results indicated that the distribution of children’s literacy skills was representative of the GRTR normative sample. The mean early literacy skills of this sample on the screener’ raw score was 11.7 (SD = 4.5; see Table 1) compared to M = 9.1 (SD = 4.3) originally reported by Whitehurst and Lonigan (2001b). Skewness of the distribution overall was near zero (−0.017, SE = 0.130), where asymmetry = 0 in the normal distribution. Kurtosis indicated a degree of flatness (−1.020, SE = 0.260), where kurtosis = 0 in the normal distribution.
Risk moderators included literacy risk and IEP status as reported by Greenwood et al. (2018), as well as DLL status. Literacy risk was established by coding cut points on the GRTR score to define three groups, where high group = 0 (if ⩽25th percentile), mid group = 1 (if >25th percentile and <75th percentile), and low group = 2 (>75th percentile). IEP and DLL risk variables were the result of binary coding of teacher-provided information, where the English, monolingual learner group = 0 and the DLL status group = 1, and where the no-IEP group = 0 and the IEP group = 1. Univariate ANOVA indicated significant differences between the GRTR means for all three risk factors (see Table 1 note). We also explored the pairwise overlap in these variables that indicated that both IEP and DLL risk were overlapping with literacy risk (DLL vs. literacy risk, φ = −.40; IEP vs. literacy risk, φ = −.18) but not each other (φ = −.03)
We analyzed the CIRCLE data, preserving activity, teacher, and child behavior co-occurrence as the focal variable (Kontos et al., 2002), by combining these three 15-s intervals into one 45-s record. The CIRCLE data set included 21,288 individual records. The momentary relationships between these events was indicated by conditional probabilities. The conventional reference to a conditional probability is in terms of the outcome of interest (e.g., CAE or TLF) preceded by the expression “given the condition,” in this case the activity or TLF, or the probability of CAE / activity = x or probability TLF / activity = x.
Conditional probabilities were computed using two-way marginal frequency tables (see Tables 3 and 4). The first research question, regarding children’s exposure to activities and TLF, is addressed in Table 3. The second research question, regarding the conditional probability of children’s behaviors (i.e., CAE, other engagement, and other behavior) given activities and TLF, is addressed in Table 4. Chi-square tests of independence were conducted to compare differences in these dependencies using alphas set to Bernoulli-corrected levels (see Table 5).
Probability of Teacher Literacy Focus Given Activity Structures.
Note. Prob = probability.
Conditional Probabilities of Child Behavior Given Activity Structures or Teacher Literacy Focus.
Note. CIRCLE = Code for Interactive Recording of Children’s Learning Environments; Prob = probability.
Pairwise Differences in the Probability of Child Academic Engagement Given Activity Structures and Teacher Literacy Focus.
χ2 = 2087, df = 5, p < .01.
χ2 = 3213, df = 1, p < .01.
To address the third research question, generalized linear mixed modeling (GLMM) was used. GLMM uniquely addressed features of the data that included lack of independence created by multiple observations per classroom (Kontos et al., 2002) and sparseness in the occurrence of some events and behaviors. The approach also enabled modeling multiple sources of dependence due to child characteristics without requiring that all events be experienced by all children to be included in the model. Additional advantages included the testing of main effects and interactions among predictors at any level of analysis, permitting the inclusion of incomplete child- or teacher-level data under the assumption of missing at random, and allowing for unbalance in the number of observations per sampling unit. The levels in the analysis were child records (N = 21,288 at Level 1), child (N = 354, intraclass correlation [ICC] = .237 at Level 2), and classroom (N = 65, ICC = .341 at Level 3) using binary outcomes and with pairwise comparisons.
The first step in the analysis estimated the extent to which CAE was predicted by co-occurring activities or TLF at Level 1. The second step estimated the extent to which these relationships varied systematically by child literacy, DLL, and IEP risk status. Because the GRTR score was a continuous variable, literacy risk was mean-centered in these moderation models. The binary outcome variable in the models was CAE versus the composite of other engagement plus other behavior, or all other behavior (not CAE). The comparison variable in each model was that least associated with the occurrence of CAE, against which differences with other variables were tested. These comparison variables were other activities and no TLF. To assist interpretation of these comparisons, we used odds ratios. Odds ratios represent the odds that an outcome will occur given a particular exposure, compared to the odds of the outcome occurring in the absence of that exposure, or (probability of A occurring / probability of A not occurring) / (probability of B occurring / probability of B not occurring) (Singer & Willett, 2003, p. 388).
Results
To What Extent Did Teachers Provide Language- and Literacy-Focused Instruction?
Teachers organized children’s preschool day with the activities reflected in the “Total” column of Table 3. This distribution was similar to that reported by Greenwood et al. (2018). The proportion of intervals that children spent in daily activities were distributed as large group (31%), center (30%), other activities (20%), small group (11%), story time (6%), and individual activities (2%). However, the proportion of intervals that teachers provided children TLF was only 0.16, or 16% (see Table 3). Children were least likely to experience TLF in the activities wherein they spent the majority of their daily time. The exception was large-group activities, where the probability of TLF co-occurring was higher than most, at only 0.26. Children were most likely to experience TLF during story time (0.63).
Was CAE Significantly Associated With Momentary Variations in Activities and TLF?
Probability of CAE given activities
The probability of CAE varied significantly given variations in the activities (χ2 = 1067.7, df = 4, p < .01; see Table 4, upper panel). CAE was significantly more likely to occur during story time, individual activities, and large group compared to small group, center, and other activities (see Table 4), replicating Greenwood et al. (2018). Inversely, children’s other engagement and other behavior were significantly more likely to occur in small group, center, and other activities. Pairwise comparisons indicated that CAE occurred significantly more often in all five activities when compared to other activities and to centers. CAE was more likely to occur during story time compared to large and small groups and individual activities. The probability of CAE between small and large groups was not significantly different (see Table 5).
Probability of CAE given TLF
The probability of CAE varied significantly depending on variation in TLF (χ2 = 1461.5, df = 1, p < .01; see Table 4, lower panel, and Table 5, lower panel). CAE was significantly more likely to occur when the teacher was focused on literacy compared to no TLF. The probability of academic engagement occurring was highest (0.61) given TLF compared to only 0.16 given no TLF (see Table 4). Children’s other engagement and other behavior were significantly more likely given no TLF (see Table 4).
Did Children’s Risk Characteristics Moderate Instruction–Behavior Dependencies?
Personal risk characteristics did moderate instruction–behavior dependencies differentially. Children in the three risk groups (literacy, DLL, and IEP) were divergent in GRTR literacy skills (Table 1), replicating Greenwood et al. (2018). Children in the high-literacy-risk group received less TLF and CAE overall (Table 6). Literacy risk status was the only risk factor that moderated conditional probabilities negatively, as in Greenwood et al. (2018). For example, children at high literacy risk were 1.0 times more likely to not be academically engaged given TLF (β = .04, p < .01) than children not at risk. The odds of children in the low-risk group being academically engaged during individual activities (β = 0.14, p < .01) were 1.2 times lower than those of children not at risk. Surprisingly, IEP moderated conditional probabilities in specific activities in a positive, not negative, direction as reported by Greenwood et al. (2018). For example, the odds of children with an IEP being engaged during center (β = 0.57, p < .01) were 1.8 times higher than those of children without an IEP, suggesting differentiation.
Differentiation of CIRCLE Teacher Literacy Focus and Child Academic Engagement for DLL Versus Mono-English Learners.
Note. AA = academic attention; AK = alphabet knowledge; AM = academic manipulation; AV = academic vocalization; CIRCLE = Code for Interactive Recording of Children’s Learning Environments; DLL = dual-language learner; O-Com = other comprehension; PA = phonological awareness; RA = reading aloud; S-Com = story comprehension; VOC = vocabulary.
DLL risk group membership acted to moderate CAE in the positive direction. DLL risk groups were significantly divergent in CIRCLE TLF overall, receiving 6.8% more TLF (M = 20.7%, SD = 16.1) than children who were monolingual English speaking (M = 13.9%, SD = 14.6), F(1, 339) = 12.673, p = .0001. Teachers were providing significantly more opportunities for DLL children to learn phonological awareness, story comprehension, other comprehension, and vocabulary (four of six CIRCLE TLF behaviors) compared to monolingual children (Table 6).
DLL risk group membership also produced 13.4% more CAE (M = 29.0%, SD = 19.3) versus monolingual English speakers overall (M = 15.6%, SD = 15.3), F(1, 399) = 14.556, p = .0001. DLL children were significantly more engaged in academic verbal response and academic attention compared to monolingual English speakers (see Table 6). In terms of specific activities, the odds of DLL children being academically engaged were 2.6 times higher in large-group (β = 0.94, p < .01) and 3.0 times higher in small-group instruction (β = 1.09, p < .01) than those of mono-English speakers. Because the DLL Status × Site interaction in analyses was not significant, differentiation favoring DLL children appeared to be occurring.
Discussion
Improving precision in MTSS-RTI decision making for children not responding to language and literacy instruction is a need in the field. To make progress on this issue, the purpose of this study was to replicate earlier findings related to CIRCLE’s sensitivity to variations in classroom instruction and children’s co-occurring academic engagement. A second purpose was examination of old and new issues in the moderation of CIRCLE’s instruction–child behavior dependencies by children’s personal risk characteristics. Current results replicated much of the first report (Greenwood et al., 2018), including the relatively uniform pattern of preschool activities that provided children with relatively infrequent exposure to literacy-focused instruction (Question 1) and similar variations in instruction–CAE dependencies given specific activities (Question 2). Results also provided important new findings related to the moderation of instruction–CAE dependencies by children’s personal risk characteristics (Question 3).
Children experienced center, large group, small group, story time, and individual activities daily at nearly the same proportions as in the first report, repeating what appears to be a uniform organization of daily activities by preschool teachers. However, in this sample, teachers provided even lower levels of opportunity to learn language and literacy (TLF) during activities, 16%, compared to 29% and 31% a year later (Greenwood et al., 2018).
As in the first report, CAE varied widely by activity structure (i.e., story time, large-group instruction) and TLF. Children spent the majority of observed time in activities that were least promoting of CAE (i.e., center and other activities). CAE was most likely occurring during story time compared to large and small groups, as in the original study. The overall proportion of children’s CAE during observations was lower in this sample (23%), as compared to 30% and 30% over 2 years in the first sample (Greenwood et al., 2018).
Children’s characteristics did moderate (strengthen or weaken) instruction–child behavior dependencies. The finding that children with literacy risk had weaker instruction–child behavior dependencies than no-literacy-risk peers was replicated. However, new findings emerged for IEP and DLL risk. Literacy risk was a negative moderator of instruction–CAE dependency in most cases; however, the dependency between individual activities and CAE was actually strengthened for high-literacy-risk compared to low-literacy-risk children. In this context, findings suggested that teachers were differentiating instruction by providing instruction of greater intensity to these higher-literacy-risk children. This was a novel finding because we had reported the opposite finding in Greenwood et al. (2018). Differentiation was also suggested for children with IEPs, in that the dependency between center activity and CAE was strengthened compared to these children without an IEP. It appeared that teachers were providing greater instructional intensity to children most in need.
Positive moderation affects were also particularly clear with respect to DLL status. In this sample, DLLs were experiencing significantly more TLF and CAE overall. Additional analyses of the CIRCLE data demonstrated that teachers were addressing DLL children with comparatively more talk about names of things (vocabulary), letters and sounds in words (phonological awareness), asking questions about happenings or events in a story (story comprehension), and talking about concepts such as categories and comparisons (comprehension). In these contexts, DLL children were significantly more likely to be talking (verbalizing) and attending to the teacher than mono-English peers. This greater CAE was most evident in the large- and small-group activities, where DLLs were significantly more likely to be academically engaged compared to their monolingual peers. Data like these were important because they described teachers who were differentiating instruction in specific situations.
Strengths of the Study
The findings are important to the field because they demonstrate the unique benefits of a measure, like CIRCLE, that is aligned with the content and standards of preschool language and literacy instruction and also sensitive to momentary instruction–child response dependencies (Greenwood et al., 2018). The work brings to the fore ways in which malleable environmental factors, including teacher behavior, affected the likelihood of child behaviors occurring that are known to be predictive of or required for early learning and achievement. The earlier work (Greenwood et al., 2018) and this replication together provide a strong basis for descriptive, nomothetic, and idiographic modifications of intervention goals of preschool MTSS-RTI language and literacy instruction.
Regarding description, the findings confirm a relatively uniform organization of the preschool classroom experience across settings and teachers and classrooms by activities in which opportunities for language and literacy learning are embedded but to a variable degree. CIRCLE description has provided greater understanding that language and literacy learning opportunities are not yet the major time focus of the preschool experience, even in programs with language and literacy outcome goals. Regarding the nomothetic understanding of the preschool classroom experiences of children, this evidence provides the field with progress toward comparatives (benchmarks) for the opportunity to learn language and literacy in preschool as well as the likely co-occurrence of children’s academic engagement overall and by different activities and teacher behaviors. These comparatives are now based on two samples of children representative of GRTR literacy normative outcomes. In MTSS-RTI, benchmarks can be used in a number of ways.
Programs interested in increasing children’s language and literacy outcomes can use CIRCLE benchmarks as baseline goals to note the need for improvement, set reasonable improvement targets, and confirm the extent to which improvement has been achieved given, for example, a change in curriculum or professional development. CIRCLE data also can be used as benchmarks for ideographic decision making as described earlier, given a child’s unique profile of opportunity to learn and co-occurring behavior, either CAE or other competing behavior. CIRCLE data may suggest the need for providing greater TLF during several daily activities. Interpretation of CIRCLE data may suggest the need for the teacher to provide more frequent prompting of child response, followed by expansions and positive reinforcement.
CIRCLE data also appear to be helpful in interpreting results of universal screening and progress monitoring (IGDI) data outcomes consistent with Tilly’s MTSS-RTI decision making model.
The work represents an innovation in our capacity to understand preschool instruction as it is actually implemented and in guiding real-time changes in instruction based on evidence, compared to the less helpful broad analyses provided by widely used preschools measures (i.e., CLASS). CIRCLE produced information needed to inform questions of why there might be an instructional explanation for a lack of progress, what might be done about it, and whether a planned change is actually being implemented. Because findings in both CIRCLE studies were based on independent, time-displaced samples representative of the GRTR’s literacy normative distribution, the reliability of CIRCLE inferences about this population of children was strengthened. This replication also adds to the trustworthiness of CIRCLE findings by extending knowledge from the smaller sample (4,729 records) to this larger sample (21,288 records).
Limitations
Several limitations were noted. Although representing a major advance in theory and use of precision information based on instruction–child behavior dependency, the data were constrained by use of momentary time sampling. This was because of its equal treatment of slower- versus faster-changing events in each 45-s co-occurrence record. An alternative point sampling of faster-changing variables nested within slower-changing variables under momentary sampling could be an improvement, as could real-time event recording. An axiom of observational measurement is that as the observer’s workload increases in number of events tracked and rates of change (complexity), reliability suffers (Dorsey, Nelson, & Hayes, 1986). The large number of variables that CIRCLE can track reliably using momentary time sampling may not be possible in live, real-time sampling because observers need to track all variables simultaneously. We argue that it is better to use CIRCLE with this limitation because its comparative advantages provide actionable information compared to current alternatives. The advantages of point sampling and real-time recording may be better suited for coding videotaped sessions outside the classroom, where researchers may code, play back, and recode behaviors.
We acknowledge that findings were correlational-descriptive, with data capable of only providing hypotheses for causality testing in experimental studies. However, such information can be appropriately used in MTSS-RTI as evidence needed in formulating hypotheses worth testing for function with individual children (e.g., Daly, Witt, Martens, & Dool, 1997). Children in this sample were observed once at midyear, and stability analyses between multiple fall occasions as was reported by Greenwood et al. (2018) were not possible.
Implications for Research
Future experimental research is needed to demonstrate that using CIRCLE in MTSS-RTI decision making actually leads to better progress in achieving literacy outcomes. Researchers can profitably use single-case experimental designs to test whether language and literacy progress-monitoring data pick up improvements resulting from CIRCLE-based changes in instruction. Research is needed that focuses on the effects of preschool instructional strategies and professional development that promote greater intentionality by embedding academic content and focus on literacy within and across daily activities (Justice & Kaderavek, 2004). Research also is needed separating CIRCLE’s instructional constructs from other data-informed guidance focused on what a child knows and needs to learn next (Al Otaiba et al., 2011). Research on the effects of strategies that differentiate instruction to intensify CAE for children with greater literacy risk, with IEPs, and who are DLLs is needed. CIRCLE data can be used to confirm the need for differentiation as well as confirm that such differentiation is being implemented.
For children with literacy risk, differentiation can include increasing opportunities to learn literacy skills. Reports indicate that these children struggle to learn phonemic awareness skills and benefit from instruction targeting these skills (Shanahan & Lonigan, 2008). These children could benefit from Tier 2 interventions, such as small-group phonemic awareness instruction (Goldstein et al., 2017). For children with IEPs, particularly those with speech, language, and literacy delays, this research is needed. Reports indicate that these children struggle to learn vocabulary skills but benefit from instructional support targeting phonological awareness and alphabet knowledge (Greenwood et al., 2013). These children could benefit from interventions such as Tier 2 small-group vocabulary instruction using automated storybook reading (Goldstein et al., 2016) or Tier 3 individualized literacy skill interventions (Kaminski, Powell-Smith, Hommel, McMahon, & Bravo-Aguayo, 2015). Differentiation for children can also include embedding oral English and vocabulary (Castro, Espinosa, & Páez, 2011) teaching strategies for DLLs given their language skill level (Goldenberg, 2008).
The issue of ecological validity will always be a consideration in planning the coverage of observations. The current CIRCLE data were based on 90-min observations of the teacher and 30-min observations of three identified children wherein observations occurred during literacy prime times, identified by teachers, where 90 min equates to 40% of half-day preschool programs. CIRCLE data did confirm that 80% of all observed intervals occurred during academic activities (Table 2), with 20% reflecting meals and snacks, cleanup and transition, and child personal care. Second, children’s 30-min observations were cycled through 10-min segments to better represent associated activities and TLF. This duration of observation should not be considered a limitation when longer and more frequent observations of individual children can be conducted. Research also is needed that focuses on improving the trade-off between precision and complexity. Can the CIRCLE taxonomy of events and behaviors be implemented live in the classroom using observational recording procedures that improve precision in the measurement of instruction–child behavior dependencies while maintaining reliability?
Implications for Practice
This report joins others confirming the need to vastly increase instructional support for language and literacy learning in preschool programs (Greenwood et al., 2013; Neuman & Dywer, 2009). Because measurable relationships exist between alterable instruction and child behavior, at least two questions now can be asked and answered by practitioners using CIRCLE: Are teachers allocating sufficient literacy focus to children more likely to respond or to children at risk? and Can we affect children’s learning trajectories on progress measures by changing instructional arrangements? These questions are self-reflective for practitioners, and CIRCLE’s data appear to support the path to solutions (see Figure 1).
Practitioners, including preschool teachers, early childhood special educators, and school psychologists interested in using evidence-based practices, should be encouraged to consider including CIRCLE data. Working with individual teachers, CIRCLE can be used to reveal the instructional interactions between teachers and children that really matter. Some routine preschool practices (e.g., story time) appear to naturally result in greater CAE and perhaps could be used more frequently and longer during the day. As with other MTSS-RTI measurement tools, comparative benchmarks will be helpful in deciding whether current performance is within the range needed for progress or if change is needed. CIRCLE benchmarks as criteria for TLF and CAE decision making based on large, diverse samples are needed.
For example, having several story time sessions or large- and small-group structured activities distributed throughout the day for approximately 15 min per session for preschool-age children could achieve this goal. Other practices (e.g., center time, other activities) that are typically longer duration but least promoting of CAE could be replaced with activities with better instruction–behavior dependencies or strengthened by embedding more effective strategies (Ziolkowski & Goldstein, 2008). These findings appear to point to a means of improving the outcomes of preschool children through greater-precision-targeted instructional experiences. Future research and practice evidence are needed to demonstrate whether this is the case.
Footnotes
Authors’ Note
The work contained herein is based on a secondary analysis of a multisite data set originally reported by 
. We want to thank the researchers and students who collected the original data: in Kansas, Maura Linas, Gabriela Guerrero, Danny Hajovsky, Joo Young Hong, Waylon Howard, Na Young Kong, Tracy McElhattan, Alyssa Pengra, and Jennifer Robertson; in Minnesota, Scott McConnell, Alisha Wackerle-Hollman, Tracy Bradfield, Michael Rodriguez, Anthony Albano, Amanda Besner, Kate Clayton, Laura Potter, Megan Rodriguez, and Braden Schmitt; in Ohio, Howard Goldstein, Sarah Downing Sean Noe, Gin Pee, Naomi Schneider, Elizabeth Spencer, and Rhonda Tabbah; and in Oregon, Ruth Kaminski, Anne Hommel, Rose McMahon, and Anna Speece. Thanks are owed to Shye Reynolds for developing and maintaining the CIRCLE observation software and to Dr. Lesa Hoffman for her advice on the statistical analyses. The authors also are indebted to the participating children, families, and preschools in Kansas, Minnesota, Ohio, and Oregon.
Collection of the original data set was supported by Grant R324C080011 from the National Center for Special Education Research, Institute of Education Sciences, U.S. Department of Education, to the University of Kansas in support of the Center for Response to Intervention in Early Childhood; Charles Greenwood and Judith Carta, co-principal investigators; and Howard Goldstein, Scott McConnell, and Ruth Kaminski, subcontracting investigators. Preparation of this manuscript was supported by Grant R324A170048 from the National Center for Special Education Research, Institute for Educational Science; Charles Greenwood, principal investigator; and the Kansas Intellectual and Developmental Disabilities Research Center (HD002528). The opinions presented in this paper are those of the authors, and no official endorsement from the Institute of Education Sciences should be inferred.