Abstract
With the advent of “Response to Intervention” there has been emphasis on preventing reading disabilities. This study examined the effectiveness of a classroom teacher Tier II intervention for struggling readers in kindergarten and first grade called the Targeted Reading Intervention. Three rural schools were randomly assigned to experimental and control conditions, with 8 experimental and 12 control classrooms. Five struggling and five non–struggling readers were randomly selected from each classroom. With the support of biweekly coaching, experimental teachers instructed struggling readers in one–on–one sessions in the classroom. Intent–to–treat analyses revealed significant kindergarten gains for initial word identification but no significant gains in first grade. Discussion focuses on the use of classroom teachers to prevent reading disabilities.
Although most children learn to read during early elementary school through a variety of effective reading instructional programs delivered by the classroom teacher, some children do not seem to profit from regular classroom instruction in reading. It has generally been acknowledged that there are two groups of children who fail to profit from regular classroom reading instruction (Foorman & Torgesen, 2001; Whitehurst & Lonigan, 1998). The first group comes to school with adequate oral language skills but has trouble with the processes involved in the relationship between oral language and the printed word. The second, larger group is characterized by problems in both oral language/vocabulary and print related/phonological knowledge. This latter group is composed mostly of low–income children who come to school without the prerequisite experiences in emergent literacy to allow them to profit from most whole class instructional practices (Snow, Burns & Griffin, 1998; Vernon–Feagans, 2009). Both groups seem to fall behind their peers in school, with a growing achievement gap over the early elementary school years (Alexander & Entwisle, 1988; Entwisle & Alexander, 1999; Morrison, Bachman & Connor, 2005; Snow et al., 1998). Children who are both low–income and come from minority families appear to be doubly disadvantaged, creating an even larger achievement gap over time (Lee & Burkham, 2002; Vernon–Feagans, 1996).
An understudied group of children who are at risk for suboptimal academic achievement are children who live in rural areas of the United States. The percentage of children in rural areas who are poor is higher compared to children from urban/suburban areas and those children who are poor are poorer than children in urban/suburban areas (O'Hare, 2009). About one third of all schools are located in rural areas, and about 20 percent of all school age children attend rural schools in this country (Provasnik et al., 2007); yet there is very little information about how these rural schools might differ from urban and suburban schools and whether specific interventions are more or less effective in rural schools. In an analysis of school entry for the children in the ECLS–K sample, Lee & Burkham (2002) found that rural children performed more poorly at school entry and had less access to high quality schools and teachers than children in the suburbs. Many rural school districts have struggled to recruit and retain highly qualified teachers (Collins, 1999; Reeves, 2003), with teacher pay about 90 percent of what urban teachers make (Provasnik et al., 2007). These factors together may put children in rural areas at risk, especially in rural areas where there is high poverty and geographic isolation. Thus, it is important to develop strategies that might aid rural schools and teachers become more effective with children at risk for reading failure.
The purpose of the present study was to investigate whether a newly developed classroom teacher professional development–reading program geared toward low wealth rural schools would produce substantial reading gains for struggling readers in early elementary school. The use of the classroom teacher to deliver the intervention was aimed to increase the cost effectiveness and efficiency of reading interventions in low wealth rural schools that have fewer resources to expend on ancillary personnel to help struggling readers.
Intervention for Struggling Readers
Research and intervention studies in a variety of contexts over the last 20 years have shown that children who are at risk for reading failure, including children from low–income families, can be helped through more explicit and intensive instruction in reading (Denton, Fletcher, Anthony & Francis, 2006; Mathes et al., 2005; O'Connor, Harty & Fulmer, 2005). Foorman and Moats (2004) summarized the elements of the most effective instruction to prevent reading failure in young children with reading difficulties. They stressed the importance of three basic elements, including: (1) explicit instruction in the alphabetic principle and related processes, while at the same time integrating these processes with reading for meaning; (2) early intervention/prevention efforts in the first few grades in school; and 3) small group/and or one on one intensive instruction. Foorman and Torgesen (2001) also stressed these factors in their review of the literature, but also included a fourth important element: (4) an effective emotional and cognitive relationship between the teacher and the child with reading difficulties, a relationship that has recently been shown to be important for academic success in the early grades for all children (Hamre & Pianta, 2005). Finally a fifth factor has been stressed by a number of studies and reviews of the literature: (5) instruction that is matched to the child's skill level (instructional match) (Foorman & Torgesen, 2001; Morrison et al., 2005; Vernon–Feagans, Gallagher & Kainz, 2010).
These successful reading intervention studies have generally used specialized teachers to deliver the intervention either within or outside the classroom. Recently, with recommendations from a national panel on “Response to Intervention” (RtI) (National Joint Committee on Learning Disabilities, 2005), there has been more focus on the role of the classroom teacher in preventing reading failure in the regular classroom via Tier I or Tier II interventions. The goal of RtI is to identify children at risk for failure early in the school years and provide them with a tiered system of interventions to prevent learning/reading disabilities. Tier I interventions are aimed at improving the quality of instruction in the regular classroom that benefits all children. Tier II interventions focus on children who do not seem to profit from high quality classroom instruction and need more specialized services within general education. Tier III interventions are conducted by a multidisciplinary team to determine eligibility for special education services, with more intensive individualized intervention. For low wealth rural schools, the training of the classroom teacher in Tier I and Tier II interventions to prevent reading failure may be particularly critical because of the lack of other resources in these schools for struggling readers.
Instructional Match using Classroom Teachers
Although previous research and reviews of the literature have found that classroom teachers were not effective in helping struggling readers in the regular classroom (Al Otaiba & Fuchs, 2006; Gersten, Morvant, & Brengelman, 1995), a number of recent studies have suggested that all children—but especially struggling readers—may benefit when classroom teachers differentiate their instruction based on children's needs. For instance, Xue and Meisels (2004) used the ECLS–K data set to examine what instructional practices might predict gains in reading for all children and children with low initial skills. They found classroom mean reading scores were significantly higher when classroom teachers used both integrated language arts and phonics more often than teachers who only used only one or both of these less in the classroom. However, children with low initial reading benefited less from integrated language arts instruction and more from phonics instruction, suggesting the importance of differentiating instruction for certain kinds of children. These findings were corroborated by other observational studies. Juel and Minden–Cupp (1998) observed that children with better reading skills made more progress in classrooms that used a meaning–based approach (instruction that helps children extract and construct meaning from text) while children with weaker skills seemed to gain more from classrooms with a code–based emphasis (explicit instruction that helps children become proficient in phonological decoding and word reading skills). Similarly, recent work on child x instruction interactions (Morrison et al., 2005) have found that children gain the most in reading when classroom instruction is geared to the children's skill levels. For instance, Connor, Morrison and Katch (2004) reported that children who entered first grade with low letter–word reading progressed more when they received explicit code–managed instruction whereas students with higher letter–word skills progressed more with meaning–based instruction. In their more recent intervention study, Connor et al. (2009) found that individualizing instruction within the regular classroom led to greater reading gains by students across the school year.
Although the studies on individualizing instruction have been important in understanding what strategies may be most important for particular groups of children, there has not been much research that has demonstrated that the classroom teacher can employ strategies at the Tier II level of “The RtI” (Gersten & Dimino, 2006; Speece, Case & Molloy, 2003). Only one recent study (Scanlon, Gelzheiser, Vellutino, Schatschneider, & Sweeney, 2008) has shown that classroom teachers can be successful in helping at risk children and this study emphasized the instructional match as critical to the success of the intervention approach. In this study, the Interactive Strategies Approach was adapted for use by classroom teachers and showed gains for struggling readers over the school year. Scanlon et al. note the importance of instructional match: “Importantly, instructional improvements were not accomplished via the implementation of a highly prescriptive program nor by the adoption of entirely new curricula but rather by encouraging teachers to analyze and respond to the instructional needs of their lower achieving students” (2008, p. 359). The current study builds on this previous work on instructional match for struggling readers by developing and evaluating a new professional development program for rural classroom teachers to help them deliver effective instruction to struggling readers.
The Purpose of the Study
The following study was designed to test the effectiveness of a new diagnostically based reading intervention, called the TRI. The TRI was delivered by the classroom teacher in the regular classroom. This Tier II intervention was specifically designed for struggling readers in kindergarten and first grade in rural low wealth communities. These rural low–wealth schools have had little access to specialized reading programs and other ancillary services. The TRI was developed to include the five elements of intervention that appeared crucial for the success of struggling readers, based on the literature just reviewed: (1) explicit phonics and word–based instruction in reading, (2) in early elementary school, (3) in one on one and small groups, (4) with an emphasis on the teacher/child relationship, and (5) with a particular stress on instruction that matched the skill level of the child (instructional match).
Unique to this intervention was the use of the classroom teacher as the primary source of delivering the intervention to children, with the support of our TRI literacy consultant and an on–site consultant.
The following questions and hypotheses were addressed in this study.
In a small randomized clinical trial, could the TRI produce greater gains in initial reading and vocabulary for kindergarten and first grade struggling readers in experimental schools compared to similar struggling readers in control schools? Was there any evidence that the kindergarten and first grade struggling readers who received the TRI were actually making greater gains in reading and vocabulary than the non–struggling readers in the same classrooms, suggesting that they were “catching up” with their non–struggling peers?
Method
The Context of the Study
Schools and Teachers
This study originally included 4 schools in a persistently poor rural county in the Southeastern United States with over 75 percent of school children eligible for free and reduced lunch. All schools were Title 1 schools but none of these schools was participating in Reading First. Schools were paired and matched based on demographic characteristics (free and reduced lunch, school size, and minority enrollment). School size varied from 357 to 402 students, with minority enrollment ranging from 50 percent to 75 percent. After random assignment, changes in administration at the school and district level led to the withdrawal of one experimental school before the study began. The 3 remaining participating schools included 20 classrooms, 8 experimental classrooms and 12 control classrooms. Teacher access to classroom aides in the experimental school varied with the school needs, such that their time in the classroom was not predictable from day to day. There was no other on–site support staff in the experimental school, including, reading teachers, school psychologists or counselors to help support the instruction of children with reading difficulties. Teacher demographics by experimental condition and grade are presented in Table 1.
Demographic Characteristics of the Teachers
Students
The overall profile of the students in the study schools can be found in Table 2. The children came from diverse ethnic backgrounds. Mother education averaged just beyond a high school education. End of grade testing (North Carolina Department of Public Instruction, 2007) suggested that all the schools were struggling to meet expectations, with children scoring on average at the 40th percentile in reading in third grade.
Demographic Characteristics of the Children
Research Design and Selection of Struggling and non–Struggling Readers
Kindergarten and 1st grade children in the experimental and control schools were initially eligible for this study if they were: (1) not diagnosed with a severe disability and (2) spoke English in the home. These criteria eliminated very few children in the classrooms.
Further eligibility was determined by the classroom teacher and one of our TRI reading consultants, using a two–step process. First, teachers administered a state–mandated kindergarten/1st grade assessment of emergent reading skills in 1:1 sessions with each child in her classroom at the beginning of the school year. The teacher was mandated to repeat these assessments every 9 weeks throughout the year. The assessments included phonological awareness, phonics (decoding), print awareness, and fluency skills. Second, based on this assessment and the teacher's knowledge of the child's reading progress over the first 6–8 weeks of the school year, our reading consultants helped the teacher rank each child in the classroom using the TRI Screening Instrument. Teachers ranked students in the classroom by whether they were: (1) profiting from regular classroom reading instruction, and (2) were below, at, or above grade level.
From among the students in each class who were rated below grade level and were not profiting from regular classroom reading instruction, 5 were randomly selected as struggling readers. From among those students rated at or above grade level and profiting from regular classroom instruction, 5 children were randomly selected to be non–struggling children. This was accomplished in both the experimental and the control schools. Thus, each experimental and control classroom contained 5 struggling students and 5 non–struggling students. Struggling children in the experimental schools received our TRI from late Fall to late Spring of the academic year, with teachers working individually with each of the 5 children for varying periods of time (see below). Struggling and non–struggling children in control schools as well as the non–struggling children in the experimental school received regular classroom instruction in reading, based on the North Carolina Course of Study. Table 3 includes the pretest and post test scores of all children on the reading and vocabulary assessments.
Fall and Spring Scores for all Children by Group
The Targeted Reading Intervention
The TRI was designed to help teachers: (a) acquire essential knowledge of early reading development that is especially critical for struggling readers; (b) learn a set of assessment based reading strategies to match the skill level of each child; and (c) apply these strategies in one–to–one and small group instruction with individual struggling readers in 15 minute daily diagnostic teaching sessions. Schools implementing these strategies did not have to change their reading curriculum but were able to use the TRI to supplement instruction for struggling readers in the regular classroom.
Each teacher used the TRI diagnostic map to match child skills to appropriate strategies and to chart the child's daily progress. As the child made rapid progress, the teacher could opt to continue the intervention in small group sessions or move on to one–on–one instruction with another struggling reader. Each 15–minute session included the following:
There are four levels of TRI Word Work from very basic levels of word identification to more advanced levels of word identification. Using teacher assessment data and help from the consultant, teachers are able to match each child's skill level with one of the Word Work levels in the TRI. From this point on, the teachers use the TRI Diagnostic map to provide a more fine grained assessment of each child on a daily basis so the child's skill level is matched with appropriate teacher reading strategies. In this way, the TRI Diagnostic map serves as both a progress monitoring instrument and a daily planning guide for instruction.
This progression through Word Work levels helps teachers understand where their students are in the process of word identification, and structures the way they choose strategies for children. For example, a child with limited alphabetic knowledge would begin at the lowest level, targeting words that can be made with the following few letters: a, s, m, t, and p. Three–letter words with beginning continuant consonants, such as “sat” or “mop” and not “top,” would be chosen because these types of words are the easiest to teach phoneme segmentation and the alphabetic principle. The teacher would use our Word Work board and three letter–sound cards (t, s, a) at the top of the Word Work board and say, “Jace, we're going to play a game today where you make a word right here (tapping lines on bottom of board). The first word is ‘sat,’/ssssat/ (as she draws her finger along the lines at the bottom of the board in concert with the sounds she is making). What do you hear here (pointing to the first line) in the word /sssat/?”
These kinds of activities are repeated with different words until the child succeeds. The teacher then goes back to her diagnostic map and develops a plan for the child's next session. At higher levels of word work, children learn to decode more difficult letter sounds within words as well as read, write and say within the context of the text.
Tri Professional Development
The TRI strategies were delivered through an ongoing, collaborative consultation model that was geared to the constraints of rural isolated schools. The TRI provided classroom teachers with a highly trained TRI literacy consultant and an on–site coordinator to facilitate the TRI implementation, using a community of practice approach that promoted teacher ownership of the TRI process that would sustain the intervention in the rural context.
Four key professional development activities facilitated the learning and the development of a community of practice for classroom teachers, professional staff in the schools and the on–site consultant: (1) a summer institute that included classroom teachers, classroom aides, and other relevant school/school district professionals, (2) biweekly collaborative consultation visits to the classroom by a highly trained TRI reading consultant, (3) weekly grade level collaborative problem–solving meetings about individual children, and (4) bimonthly two–hour professional development sessions designed to meet the needs expressed by the classroom teachers.
The 3–day summer institute introduced teachers and professionals to the TRI content through interactive large and small group sessions that included practicing the strategies and using problem solving strategies with case studies of struggling readers. Small group discussions laid the groundwork for the preparation of classrooms during the fall so the TRI could be implemented in January.
The weekly/biweekly TRI literacy consultant visits had two foci: (1) modeling, coaching and problem solving with teachers as the TRI was implemented with their struggling readers and (2) supporting the on–site school consultant as she assumed responsibilities of the consultation process. In the current study, because of the paucity of on–site professional staff, a local retired elementary teacher served as the school's on–site (part–time) consultant. Creating a professional learning community that problem solved about individual children was facilitated by weekly meetings of 30 minutes or less, focused on TRI implementation with struggling students, case analysis of TRI students and planning TRI instruction for students. Periodic meetings by TRI staff with the principal, school board, and other school district staff were meant to encourage the commitment needed for the success of the intervention. These kinds of activities have been shown to be particularly effective in creating sustainable teacher change that is supported by a community of practice professionals, especially important in rural communities (Buysse & Wesley, 2006; Stigler & Hiebert, 1999; Vernon–Feagans et al., 2010).
Finally, the TRI professional development process also provided ongoing, integrated professional development for teachers on a monthly basis. The monthly two–hour sessions were based on what the teachers thought would be helpful in the implementation of the TRI, including advanced discussions of TRI components, new ideas for extending the learning during independent work, and honing the diagnostic thinking process (authors, under review).
In the fall of the school year, teachers participated in workshops and the TRI consultants worked with individual teachers to help them prepare for the implementation of the TRI. This involved problem solving sessions that allowed teachers to arrange and manage their classrooms so that they would be able to work one on one with individual children. The shortage of available classroom aides and the teachers’ limited experience with student independent work strategies presented major barriers to the implementation of the TRI until late fall when teachers were equipped with TRI skills and classroom management strategies.
Procedure and Measures
All children in the study were administered a battery of standardized tests in the fall and again in the spring of the school year. Teachers filled out questionnaires about their professional background and classroom. All child assessments were done in the schools in a quiet room. Graduate students from the University of North Carolina at Chapel Hill conducted the child assessments. The assessors had previous testing experience and participated in a two–day training, which included the administration of the complete battery with non–participating students. Assessors were not informed as to which schools were experimental or control. The following measures were administered to children in the fall and the spring.
The Woodcock–Johnson Tests of Achievement, III (WJTA, III) (Woodcock, Mather & Schrank, 2004). Two subtests of the WJTA, III (2004) were administered to all children. Word Attack measures skill in applying phonic and structural analysis skills to the pronunciation of unfamiliar printed sounds and words. The initial items require the child to produce the sounds for single letters. The remaining items require the child to read aloud letter combinations that are phonetically consistent, or regular, patterns in English orthography but are non–words or low–frequency words. The items become more difficult as the orthographic complexity of the non–words increases. Word Attack has a median reliability of. 87 in the 5 to 19 age range (Woodcock, Mather & Scrank, 2004). The Letter–Word Identification subtest measures the child's word identification skills. The initial items require the child to identify letters that appear in large type on the subject's side of the test book, and the remaining items require the child to pronounce words correctly. The child is not required to know the meaning of any words. The items become increasingly difficult as the selected words appear less and less frequently in written English. Letter–Word Identification has a median reliability of. 91 in the 5 to 19 age range (Woodcock et al., 2004).
Raw scores were converted into W scores, which are a special transformation of the Rasch ability scale. The W scale has mathematical properties (e.g., equal interval units) that make it well suited for use as an intermediate step in the interpretation of test performance and especially useful for interpreting gain scores.
The Peabody Picture Vocabulary Test–Third Edition (PPVT–III) (Dunn & Dunn, 1997) is an individually administered, norm–referenced test of receptive vocabulary knowledge. Children are asked to select a picture, from among four black–and–white illustrations presented that best represents the meaning of the stimulus word presented orally by the examiner. Raw scores are determined by subtracting the number of errors above the basal from the ceiling item total. Raw scores are then converted to a percentile score using a table corresponding to the child's age. Alpha coefficients for the PPVT–III for elementary age students range from. 92 to. 95.
The TRI Fidelity Measure assessed the quality and duration of TRI instruction for each Experimental Struggling child from November to May in order to monitor whether these children were receiving consistent and quality intervention. We developed two separate 5–point Likert scales. The first assessed the duration of TRI practices, and the second assessed fidelity quality of TRI practices. The duration of TRI was derived from reports by teachers in their regular meetings with TRI consultants. The scale reflected the total number of weeks an experimental struggling child received TRI–specific literacy practices. A rating of “1” indicated no weeks of TRI; “3” indicated 4 to 9 weeks of total TRI; and “5” indicated 19 weeks or more of total TRI. The quality of intervention scale rated the classroom teachers’ use of diagnostic assessing/planning tools, and faithfulness to TRI strategies for each struggling student as assessed by the TRI literacy consultant notes from each consultation session. In making the ratings of individual classroom teachers, the TRI consultants used anchor points to guide the ratings. These included: regular use of the TRI (at least 15 minutes/4 times per week), the use of diagnostic assessing/planning tools, and the faithfulness to the TRI strategies. A score of “1” indicated little or no instruction akin to the TRI, and “3” indicated at least moderate fidelity of the TRI instruction at least twice a week, using at least two of the three components (fluency, word work, or guided oral reading), using some of the diagnostic tools available in the TRI. A rating of “5” indicated high quality TRI instruction at least 4 times a week, including all three major components, with consistent use of the TRI diagnostic tools.
TRI consultants completed the TRI Fidelity Measure in the spring using observation and weekly records on classroom teacher implementation. The Duration of Intervention and Quality of Intervention scales were highly correlated (r= 0.89), and thus were combined to form a Total Fidelity variable that was the average of the two scales combined. Examination of the fidelity scores revealed high implementation for children in kindergarten but low levels of implementation for teachers in first grade. (See Table 4).
Mean Fidelity Ratings by Grade
Results
Analysis Strategy
The analysis strategy was an “intent to treat” (ITT) approach. All kindergarten and first–grade classrooms in all three schools were included in the analysis, regardless of the teachers’ investment and cooperation with our intervention. The intent to treat analysis included two–level hierarchical models of children's gains in literacy across one year. Separate analyses were conducted by grade. The models estimated separate intercepts for classrooms to adjust standard errors in relation to the nested structure of the data. We did not estimate separate intercepts for schools because of the small number of participating schools (n= 3). At level one we included fixed effects for treatment, child gender, and mother's level of education. Treatment was coded as a four–level variable indicating random assignment to one of four groups: Control non–struggling, Control struggling, Experimental non–struggling, Experimental struggling. Mother's education and child gender were centered for analysis. We followed these models with two contrasts: comparison of gains for struggling experimental versus struggling control (ITT); and comparison of gains for struggling experimental versus non–struggling experimental (catch–up).
To estimate the change score models, we used a mixed models approach in order to account for the separate sources of variance due to children being nested within classrooms. These models insured that the standard errors of the parameter estimates were correct (Raudenbush and Bryk, 2002). The effect sizes were calculated using a method for clustered analysis detailed by Raudenbush and Xiao–Feng (2001). Standardized effect sizes were calculated by dividing the mean difference between groups by the square root of the total variation for the model. These are scaled like a d–type effect.
Kindergarten intent to treat analyses of the Word Identification, Word Attack, and the PPVT (See Tables 5, 6 and 7) revealed that the experimental struggling children gained significantly more from fall to spring on Word Identification, with an effect size of 1.03. There were no other significant intent to treat effects.
Letter Word Identification: Kindergarten
Word Attack: Kindergarten
Peabody Picture Vocabulary: Kindergarten
Kindergarten contrasts to examine whether the experimental struggling children (struggling readers) gained significantly more than their non–struggling peers (non–struggling readers) in the same classrooms revealed a significant catch up effect for Word Identification, with an effect size of. 77. There were no other significant catch up significant effects on the other measures.
First grade intent to treat analyses of the Word Identification, Word Attack, and the PPVT (See Tables 8, 9 and 10) revealed no significant effects and no significant catch up contrasts.
Letter Word Identification: First Grade
Word Attack: First Grade
Peabody Picture Vocabulary Test: First Grade
Discussion
The aim of this study was to evaluate whether classroom teachers could implement a one–on–one intervention in the regular classroom that could benefit struggling readers in kindergarten and first grade. The results of this small randomized control trial suggested that the TRI, delivered by classroom teachers in one–to–one 15 minutes sessions over the course of more than a semester, could benefit the word reading skills of kindergarten struggling readers. In addition, there was evidence that kindergarten struggling readers were catching up to their non–struggling peers, gaining double the number of W score points in comparison to their non–struggling peers over the school year. However, there were no effects of the intervention in first grade.
In kindergarten, the TRI findings support previous research that has found that reading specialists who work with struggling readers outside the classroom in early elementary school can produce significant reading gains for these readers (Denton et al., 2006; Mathes et al., 2005; Morris et al., 2000; Torgesen et al., 1999; 2001). This study contributes additional knowledge by demonstrating that the regular classroom teacher can also produce reading gains in struggling readers through a relatively modest investment of time with struggling readers. Our findings suggest a very cost effective way to enhance teacher classroom reading practices without using reading specialists to deliver the intervention.
Our findings are in contrast to other previous professional development programs in reading for classroom teachers that have not been found to be particularly effective (See review by Al Otaiba & Fuchs, 2006). One of the largest studies of teacher professional development in reading in early elementary school has recently been completed through an IES randomized clinical trial (Garet et al., 2008). This large study provided classroom teachers with in depth knowledge about teaching reading and the power of direct instruction for struggling readers. Although the professional development program did improve teacher skills, knowledge, and direct instruction in the classroom, these teacher improvements did not translate into reading gains for struggling readers. This recent evidence appears to suggest that we need other professional development strategies to help classroom teachers improve the reading of at risk readers.
The failure of previous studies to find that classroom teachers can be effective in helping struggling readers may have been because teachers were not instructed to work one–on–one with children and/or to carefully match their teaching strategies to the skill level of the child. Our TRI may have been effective in kindergarten because the teachers were instructed in and actually implemented a carefully crafted diagnostic teaching cycle for each individual child that was quite different from what the teachers were doing with other children in the classroom. Our results support the more recent work that suggests that when regular classroom teachers match their instruction to the skills of the child in the regular classroom, those children make the most reading progress in early elementary school (Morrison et al., 2005; Connor et al., 2004, 2009). For children who are struggling readers, it seems particularly important that teachers not only assess the skill levels of children but that they have the appropriate instructional strategies that can help that child progress in reading. Thus, even though the Mathes et al. (2005) study found that their enhanced classroom teacher intervention did not help children through the provision of assessment data from the Texas Primary Reading Inventory (TPRI), this instrument was not linked to fine grained teacher strategies that matched the specific assessment profiles of students. Thus, teachers might not be able to take the assessment information and be able to craft the appropriate reading strategy without explicit coaching to help them understand these links. The TRI, on the other hand, made a direct link between fine distinctions in assessment and appropriate teacher instructional strategies for each child.
The TRI was also implemented for less time than most other interventions for struggling readers and yet had positive effects for kindergarten students. Most of those studies of early reading intervention entailed more time per day, usually 20 to 60 minutes over the course of months and/or years. In our intervention the adequate fidelity children received 15 minutes per day for more than 9 weeks per child over the course of the school year. Although the effect sizes in the current study were not as great as for some of the other long–term intervention studies (probably because of the lesser time), our results were fairly comparable to many of the other reading studies for struggling readers that provided services outside the regular classroom with a specialized professional for a limited amount of time (Torgesen et al., 1999, 2006; Morris et al., 2000; Mathes et al., 2005). Thus, even though teachers only worked 15 minutes a day 4 days a week with each child for part of a semester, we found significant effects on word identification as well as gains that were greater than the non–struggling readers. This was probably due in part to the literacy coach who was available to the teacher in the regular classroom to help problem solve about each individual child on a biweekly basis. This coaching was a critical part of the intervention that needs greater scrutiny as the crucial mediator of the intervention effectiveness.
We did not find effects of our intervention in first grade. Although it was possible that the TRI just did not work as well in first grade, it is more likely that the fidelity data reported in Table 4 are implicated in our failure to find effects. Although we cannot make a causal link, it appeared that the first grade children received low levels of implementation fidelity (mean of 2.15), suggesting inadequate quality of intervention. This kind of score was indicative of teachers who did not implement regularly and thus was related to lower reading gains for their children. Lipsey and Cordray (2000) stress the importance of understanding these implementation issues, even within the framework of an experimental study. Thus, we will discuss our findings relative to fidelity, especially since we used the classroom teacher to implement the intervention.
Fidelity of implementation has always been important to measure in intervention studies. Those studies that employed their own intervention specialist teachers, typical of most of the studies reviewed here, were almost always able to achieve very high levels of fidelity probably because they had control over what their specialists did in the school. On the other hand, when classroom teachers deliver the intervention and they are not financially compensated for delivering the intervention, the study personnel have much less control over the fidelity of intervention. For instance, Connor, Morrison, Fishman, Schatschneider, and Underwood (2007) used classroom teachers and found that the children who gained the most in their A2i intervention were those whose teachers who implemented the intervention with the highest fidelity. In a study of the effects of the Voyager reading program in regular classrooms, the fidelity of the implementation was significantly related to the outcomes for children (Frechtling, Zhang & Silverstein, 2006). In the most recent Social Policy Report from the Society for Research in Child Development (Ludwig & Phillips, 2007), the benefits of Head Start were estimated by examining those children who actually received Head Start and therefore received the treatment as planned. The effect sizes for the comparison between the experimental and control group on the Woodcock–Johnson were 50 percent greater when the analysis took into consideration the children who actually received the Head Start treatment and thus had adequate fidelity. In our study, we failed to motivate and engage the first grade teachers to implement with high fidelity. Our first grade teachers reported a number of reasons for why they were not able to implement with high fidelity, even after we have tried a variety of strategies to help them. Further research needs to be undertaken to better understand how to motivate classroom teachers to implement faithfully and to help convince school administrators of their support of teachers in participating in educational interventions like the TRI.
Finally, although this study was encouraging in understanding the effective role of the classroom teacher in helping struggling readers, it has a number of limitations that need to be addressed in future research. First, the small sample size for random assignment allowed for undetected selection bias. Even though we did matching and used control variables, larger studies using this intervention are needed to corroborate our initial findings. Future studies with classroom teachers need to examine more effective ways to motivate teachers to actually implement Tier II interventions faithfully so that fidelity is high across all teachers. This may be especially important in low wealth schools, where classroom teachers bear almost all the responsibility for teaching children with few other professional staff available for helping them and their children. We also did not have detailed ongoing fidelity data, but only summary ratings by the TRJ coaches. Future research needs to carefully document what components of intervention are most difficult to implement for classroom teachers. In addition, the TRI did not have significant effects on decoding (word attack) and vocabulary that would be important for further improvements in children's reading. It would be hoped that with a longer intervention period, effects might have been detected on these other measures. Given the small sample size in this study, we were also not able to examine subgroups of students who profited most and least from the intervention in order to help us understand for whom and under what conditions the intervention was most effective. These interactions with treatment would be important to pursue in future studies. Lastly, it would have been helpful to understand whether these gains could be sustained over time so that the struggling readers might catch up completely with their non–struggling peers.
