Abstract
This study investigated the relative effects of three treatments with varying instructional emphases in reading with a comparison condition. Eighty-seven students in fourth grade with reading impairments were assigned through stratified random assignment to one of four conditions: (a) comprehension emphasis, (b) word study emphasis, (c) emphasis of either comprehension or word study based on the student’s pretest reading profile, or (d) school-provided intervention comparison condition. Students in the three researcher-provided treatments received intervention in small groups with a trained tutor for 30 min daily for approximately 28 weeks. Results revealed no statistically significant main effects between conditions on measures of word reading, fluency, vocabulary, or comprehension. Students with limited English proficiency performed significantly better at posttest in all conditions than other students. Discussion addresses the challenges of successfully remediating reading problems with older students with significant reading problems.
Considerable research conducted over the past 30 years provides extensive knowledge regarding early intervention for students with reading difficulties (e.g., Jenkins & O’Connor, 2002; Torgesen et al., 1999; Vellutino et al., 1996). The highest student effects result when explicit, systematic instruction is provided in both foundation skills such as phonological awareness and phonics as well as higher level reading tasks, such as fluency, with increased attention to word meaning and understanding text (vocabulary and comprehension; National Reading Panel, 2000). Using this body of knowledge to design effective instruction for students in Grades K–3 is associated with reducing the incidence of reading difficulties (Torgesen, 2000).
Despite these successes with early reading interventions, there are still many students entering fourth grade who struggle significantly with reading (National Center for Education Statistics [NCES], 2005). The problem of older students with reading impairments is significant for several reasons. First, students who continue to struggle with reading after third grade have particular challenges accessing the general education curriculum not only in reading but also other content areas such as social studies and science where mastery of reading is often expected for building background knowledge and learning new information. Second, these reading challenges do not dissipate over time. Students who enter the upper grades as poor readers continue to struggle throughout their schooling (Francis, Shaywitz, Stuebing, Shaywitz, & Fletcher, 1996) putting them at high risk for academic failure, drop out, unemployment, overall low income, and even poor health (NCES, 2004, 2006). Nationally, the number of students served in special education with a learning disability nearly doubles (increases by 99%) in the upper elementary grades (U.S. Department of Education, 2010). Students with reading impairments make up the largest percentage of students with learning disabilities (Fletcher, Lyon, Fuchs, & Barnes, 2007). Thus, there are many students struggling with reading after third grade who are at risk for identification with a reading disability.
Biancarosa and Snow (2004) suggested that most older, struggling readers demonstrate difficulties with reading comprehension, particularly with the complex texts presented in the upper grades. As a result, interventions addressing skills for increasing knowledge and understanding of text are typically recommended. For students with reading difficulties and disabilities, these advanced skills may assist them in accessing the general education curriculum more effectively; however, it may not solve some of their more basic problems of reading unknown words, including multisyllabic words. There is considerable evidence that students with reading difficulties may struggle with basic reading skills such as word recognition and fluency (Leach, Scarborough, & Rescorla, 2003). The lack of mastery of basic reading skills could significantly affect a student’s ability to read and comprehend complex texts. A fundamental issue for teachers of students with reading difficulties in Grade 4 and higher is information regarding how much instructional emphasis should be provided for word reading activities and meaning-based activities. At this time, there is little or no research to guide teachers in determining the impact of multicomponent reading interventions that emphasize advanced word study compared with an emphasis on comprehension instruction.
To better understand the relative effects of varying instructional emphases on the reading outcomes for upper elementary students with reading impairments, we implemented two multicomponent—but conceptually different—treatments, one with a comprehension emphasis focusing on improving student understanding of text with explicit comprehension instruction and a second intervention with a word recognition emphasis focusing on improving student’s accuracy and fluency of word identification and text reading. To further determine the relative influence of these instructional emphases, we examined a third treatment group of students who were provided either the comprehension or word reading emphasis intervention based on their pretest reading profile so that treatment would be more carefully aligned with their reading needs.
Research in Comprehension
In the upper elementary grades, teachers’ expectations increase for independent reading and learning by students, and instruction on learning how to read is deemphasized. For students with reading disabilities, explicit instruction in comprehension with many opportunities to practice reading and understanding text may be the most appropriate instruction for improving student learning and assisting students with the complex content area text they are expected to read in the upper elementary grades (e.g., Biancarosa & Snow, 2004).
Although difficulties with word reading that might be addressed through decoding or word recognition can impact students’ reading comprehension, there are other causes for comprehension deficiencies, including weak concept knowledge and difficulties monitoring comprehension (Garner, Alexander, & Hare, 1991). When students increase their concept knowledge and understanding of text, they can draw on more complex ideas and can read more text. This text practice and understanding may also lead to better automaticity in reading.
We recently synthesized the research on upper elementary reading interventions for students with reading difficulties and disabilities (Wanzek, Wexler, Vaughn, & Ciullo, 2010). Only nine experimental studies were found. The largest number of experimental studies (n = 5) addressed reading comprehension and/or vocabulary development yielding moderate to large effects on comprehension outcomes. Practices including previewing text and connecting with student knowledge, use of self-questioning and self-regulating while reading, and summarizing what is learned from text were examined in the interventions. Although findings were positive for students participating in the interventions, all of the studies used researcher-developed outcome measures aligned with the intervention. None of the studies reviewed in the synthesis measured the effects of comprehension instruction with standardized comprehension measures.
Research on Word Recognition
As students advance in school, reading demands increase, and thus a major focus for teachers and researchers alike has been finding ways to improve concept knowledge and reading comprehension across text types (Baumann & Kame’enui, 2004). However, many older students with reading impairments have not mastered the basic reading skills (i.e., word reading with accuracy and fluency) needed to effectively benefit from instruction that focuses solely on reading for meaning (Leach et al., 2003). For students with disabilities, continued instruction in decoding and word recognition skills may be particularly important given that a major cause of reading disabilities is a weakness in phonological processing (Liberman, Shankweiler, & Liberman, 1989), even in adults with dyslexia (Ransby & Swanson, 2003).
Evidence from studies with older students who exhibit deficits in decoding and fluency have suggested that these students must receive instruction in the basic elements of word reading, regardless of how old they are (Abbott & Berninger, 1999). Moreover, student improvement in reading words is associated with high effects in reading comprehension (McCandliss, Beck, Sandak, & Perfetti, 2003; Vaughn, Gersten, & Chard, 2000). In several studies examining the effects of interventions for students in the upper grades, students receiving interventions with a focus on phonological activities demonstrated increased reading outcomes not only on word recognition but also on comprehension (Bhattacharya & Ehri, 2004; Torgesen et al., 2001).
In our recent synthesis of the literature on upper elementary reading interventions for students with reading difficulties and disabilities, studies examining word study interventions that assisted students in learning to map the sounds of language to letters and words were associated with small to moderate effects on word recognition, fluency, and comprehension outcome measures (Wanzek et al., 2010). In contrast to the comprehension interventions described earlier that may have inflated effects due to the administration of researcher-developed outcome measures, the three experimental word recognition studies administered norm-referenced measures at posttest.
Importantly, only two studies in the synthesis implemented more than one reading component in the intervention. Only one study in the synthesis examined a multicomponent intervention that included aspects of word recognition and comprehension. High effects were noted on standardized measures of word reading, fluency, and comprehension for the multicomponent interventions (Wanzek et al., 2010). Additional research is needed to examine multicomponent interventions at the upper elementary level. It is clear that implementing interventions with multiple components show promise at the upper elementary grades and, certainly, these types of interventions have been successfully implemented at other grade levels. However, the research focus on individual strategies in only one reading component at the upper elementary level suggests the need for further research on multicomponent interventions and specifically ways in which teachers can make decisions about how to organize multiple components of instruction in a time-limited intervention. Therefore, in this study we sought to examine multicomponent interventions with emphasis in different components.
Current Study
In addition to examining the differentiated effects of an intervention with an emphasis on advanced word recognition instruction, and an intervention with an emphasis on explicit comprehension instruction, we examined the effects of a third condition designed to be responsive to students’ needs in word recognition or comprehension. In this condition, students with a weakness in word recognition received the word recognition emphasis intervention while students with adequate word recognition but comprehension difficulties received the comprehension emphasis intervention. We hypothesized that matching the emphasis of instruction to student needs may enhance outcomes for students with reading disabilities in the upper elementary grades. In this study, all of the interventions included instruction in word recognition, vocabulary, fluency, and comprehension and differed only in the emphasis and time devoted to specific components of instruction.
The purpose of this study was to examine the relative effects of three treatments with varying instructional emphases in reading with a comparison condition in fourth grade. In any intervention research, student level factors can provide sources of variation that affect outcomes. Previous research has suggested culturally and linguistically diverse students underachieve in our schools (e.g., Al Otaiba et al., 2009). Given the diverse sample that participated in this study, we were also interested in examining whether there were moderating effects of English proficiency.
Method
Participants
The study was conducted in five schools in one school district in the southwestern United States serving large numbers of students from economically disadvantaged and culturally/linguistically diverse backgrounds. We implemented a set of standard selection criteria to allow a clear definition of a sample of students with impaired reading who were at high risk for identification of disability based on their significant difficulties in understanding text and, thus, allowing generalization of the results of the experimental study. Given the large numbers of students who are identified with a learning disability in the upper elementary grades, we sought a sample of students demonstrating reading achievement well below grade-level expectations whose low achievement after 4 years of reading instruction may demonstrate risk for a learning disability.
Grade 4 students with reading difficulties from the district’s English classrooms were identified through a two-step screening process:
Classroom teachers were asked to indicate all students (a) identified with dyslexia or a learning disability in reading; (b) receiving supplemental reading instruction, Title I services, or school tutoring; or (c) reading below grade level.
These nominated students were screened for inclusion in the study using the comprehension subtest of the Gates-MacGinitie Reading Tests (MacGinitie, MacGinitie, Maria, Dreyer, & Hughes, 2006). Students scoring at or below the 25th percentile on the comprehension subtest were included in the study.
Although many studies of students at risk for reading difficulties have selected students below the 30th percentile (e.g., O’Connor, Harty, & Fulmer, 2005; Vellutino, Scanlon, Small, & Fanuele, 2006), the school district considered students below the 25th percentile as part of their prereferral process; thus, we chose to work with students with more severe impairments and at higher risk of identification with a disability. A total of 101 students were identified for the study and scheduled for intervention. Over the course of the study, 14 students moved or withdrew from the study, yielding a total of 87 students with pretest and posttest data for analyses. Of these 87 students below the 25th percentile in reading comprehension, 23 students also demonstrated scores below the 25th percentile on the word recognition measure and 7 additional students demonstrated scores below the 25th percentile on the word attack measure.
Demographic data for the sample is presented in Table 1. No significant differences between the study groups were noted for any of the demographic variables. The total sample included 54 boys and 33 girls. Eighty-six percent of the sample was Hispanic, and 90.8% of the sample was enrolled in free or reduced-cost lunch programs, consistent with the overall district demographics. We collected the demographic data from the school district, along with information on limited English proficiency (LEP). The district identifies students with LEP based on a school-administered standardized test of language proficiency.
Sample Demographics
Note: AA = African American; SLD = specific learning disability; LEP = limited English proficiency.
n = 21.
n = 24.
n = 19.
n = 23.
Procedures
Students identified for the study were administered a battery of assessments by the research team at the beginning of the school year. The students were then assigned to one of four study conditions through stratified random assignment based on a rank ordering of students on the screening measure: (a) word recognition emphasis, (b) comprehension emphasis, (c) responsive emphasis, or (d) comparison. The pretest scores for the students randomly assigned to the responsive emphasis group were further examined to determine whether they would receive the word recognition or the comprehension emphasis intervention. The students in the responsive group who scored at or below the 25th percentile at pretest on the Letter-Word Identification (LWID) subtest of the Woodcock-Johnson Psycho-Educational Test Battery-III (WJ III; Woodcock, McGrew, & Mather, 2001) were provided the word recognition emphasis intervention (n = 8; LWID M = 86.63; Passage Comprehension M = 83.38). The students in the responsive group who scored above the 25th percentile at pretest on the LWID subtest were provided the comprehension emphasis intervention (n = 11; LWID M = 96.55; Passage Comprehension M = 83.18).
Students in each of the treatment conditions (word recognition emphasis, comprehension emphasis, or responsive) were grouped into small instructional groups of 2 to 4 students within their respective schools and provided intervention by a teacher hired by the research team. Intervention was provided in daily 30-min sessions throughout the school year (28 weeks of instruction). The total number of intervention sessions attended by individual students ranged from 85 to 114. Students randomly assigned to the comparison group did not receive any of the research-provided interventions and continued in school-provided interventions. Following the completion of interventions, all students were assessed with the complete battery of measures. All posttest data were collected within 2 weeks of the last session, with equal numbers of students from each study condition tested each day.
Researcher-Hired Teachers
Six teachers were hired and trained by the researchers to provide intervention for the treatment groups. All teachers were women, had at least a bachelor’s degree in education (four had a master’s degree), and had previous teaching experience ranging from 3 to 20 years (average of 8 years).
The teachers participated in approximately 40 hr of training (20 hr on the word recognition emphasis and 20 hr on the comprehension emphasis) over a 2-week period prior to the start of the interventions. Each teacher was assigned equal numbers of instructional groups in word recognition emphasis and comprehension emphasis to control for teacher effects.
Interventions
The multicomponent interventions differed in the emphasis on word recognition or comprehension components. Time spent in text reading (8–10 min) and vocabulary instruction (3–4 min) was held constant across the treatment conditions.
Word recognition emphasis intervention
Students assigned to the word recognition emphasis treatment group received instruction in which a large percentage of time was devoted to teaching students how to read words and apply that knowledge to text. The third edition of the Wilson Reading System (Wilson, 2002) was implemented throughout the intervention. The Wilson Reading System is a widely used reading intervention for students in Grades 3 and above. Significant gains in student outcomes have been noted for the multicomponent intervention with students with learning disabilities in Grades 3 through 12 on measures of word attack, comprehension, and spelling after 1 year of implementation (Wilson & O’Connor, 1995). The word recognition aspect of the program has also been associated with high effects on word reading measures (Torgesen et al., 2006).
The Wilson Reading System contains 12 steps. Lessons in each step are completed until mastery levels are reached. Mastery is measured daily with untimed word-reading measures included with the program. Steps 1 and 2 of the program focus on phoneme segmentation and blending of sounds. Step 3 focuses on multisyllabic words and breaking words into syllables. Steps 4 through 6 teach VCe, open syllables, suffixes, and consonant –le syllables. Steps 7 through 12 emphasize advanced word recognition and complex word types with strategies for application in controlled and uncontrolled text. The Wilson Reading System provides materials for ample practice opportunities in word reading, spelling, sentence reading and writing, and connected text on a daily basis. In addition, cumulative review is built into each part of the lesson. We implemented the alternate day program option with the decoding and encoding activities included in the Wilson lessons implemented on two, separate consecutive days, with text reading addressed each day.
Although the emphasis was on word recognition, the intervention integrated and included instruction in all essential elements of reading for students. Vocabulary words from the text were introduced in isolation with basic definitions and explanations before reading the text. Comprehension checks occurred in each lesson after reading the text and included questioning, visualizing, and retelling of the text included.
Comprehension emphasis intervention
Students assigned to the comprehension emphasis treatment group received reading instruction in which a large percentage of time was devoted to learning comprehension skills and strategies. We implemented Collaborative Strategic Reading (CSR; Klingner, Vaughn, Dimino, Schumm, & Bryant, 2001) using expository science text that aligned with the science topics being covered in the classroom. Significant gains in reading comprehension have been noted for students with reading difficulties and learning disabilities, including English language learners, after CSR participation (Bryant et al., 2000; Klingner & Vaughn, 1996; Klingner, Vaughn, & Schumm, 1998). In contrast to the word recognition emphasis intervention, CSR devotes more time to instructional strategies in reading comprehension.
As part of CSR, reading comprehension strategies were taught in four groups: Preview, Click and Clunk, Get the Gist, and Wrap Up. Students implemented the Preview strategy before reading text by brainstorming and predicting. Click and Clunk was a strategy used during reading to self-monitor comprehension. Within the Click and Clunk strategy, students were taught four fix-up strategies for use when they came across words or sentences they needed to know more about to comprehend the text. Get the Gist was also used during reading when students identified the most critical information (main idea) in the paragraph or section of text they read. Students applied the Click and Clunk and Get the Gist strategies multiple times during the reading of each text. The Wrap Up strategy was employed after reading when students generated and answered questions about the text and reviewed the most significant ideas in the passage. While applying each of the comprehension strategies, students completed learning logs to reinforce use of the strategies. Students learned each strategy one at a time with daily, cumulative practice in applying previous learned strategies while reading text.
Although the emphasis was on comprehension strategies, the CSR intervention integrated and included instruction in all essential elements of reading for students. As with the word recognition emphasis intervention, vocabulary words from the text were introduced in isolation, with basic definitions and explanations, before reading the text. Word recognition strategies were addressed through the Click and Clunk strategy when students learned and applied fix-up strategies for unknown words. Students were taught to identify affixes and to try breaking multisyllabic words into smaller words. The text-reading portion of the comprehension emphasis intervention mirrored the word recognition emphasis, with about 10 min of the students’ time spent reading text.
Responsive emphasis intervention
Students randomly assigned to the responsive emphasis were placed in instructional groups according to their area of deficit based on pretest measures. Students demonstrating a deficit in word recognition (at or below 25th percentile on the WJ III: Letter Word Identification subtest; Woodcock et al., 2001) received the word-recognition emphasis intervention. Students in the responsive group who qualified for this study based on comprehension difficulties but demonstrated adequate word recognition skills received the comprehension emphasis intervention.
Comparison group
The school comparison group continued to receive instruction and intervention provided by the school. We collected data on any supplemental school interventions provided to the students in the comparison group through teacher interviews. Eight students (34.8%) in the comparison group received one supplemental intervention. Three additional students (13.0%) received two concurrent supplemental interventions during the school year. Of these 11 students, 9 received between 200 and 360 min of intervention per week. The other 2 students received 25 and 60 min of supplemental intervention per week, respectively. The supplemental school interventions were provided in small groups of 2 to 3 students, with 1 student receiving one-to-one instruction. The school supplied supplemental interventions to students who were identified as at risk for failing the state test. Teachers providing the school intervention indicated a focus on test-taking skills specific to reading passages and answering questions. The teachers indicated this instruction included decoding unknown words, identifying the meaning of unknown words from context, selecting main ideas, and locating pertinent information in text to address questions.
Fidelity of Implementation of the Intervention
Weekly observations were conducted for each researcher-hired teacher who was implementing the treatment interventions. Feedback was provided at each observation to ensure fidelity. Weekly meetings were also held to address teacher needs, student progress, and implementation issues. In addition, to document intervention fidelity, we collected fidelity of implementation data once a month for each teacher via the Implementation Validity Checklist.
The fidelity of implementation instruments addressed whether the treatment was implemented as planned. Fidelity of implementation was measured by means of a set of item types (e.g., checklists and rating scales) that had a common structure but with items that were unique to each reading intervention approach. The items were specific to the architecture of the intervention and reflected the critical elements of each intervention type. For each statement, the observer indicated the extent to which the teacher implemented that aspect of the intervention on a Likert-type scale (e.g., 0 = not implemented, 1 = weak implementation, 2 = adequate implementation, 3 = excellent implementation). Complementing treatment-specific items, Quality of Implementation addressed how well the intervention was implemented on a 3-point Likert-type scale—exploring such issues as features of instructional quality, including student engagement, time on task, lesson pacing, and use of corrective feedback.
The fidelity data, as well as the ongoing researcher observations, revealed that intervention implementation and quality of instruction were high throughout the study. Mean implementation ratings and quality ratings for all teachers were above 2.75 (of a possible 3.00) for the word recognition emphasis intervention. For the comprehension emphasis intervention, mean implementation ratings were above 2.50 for all teachers and quality ratings were above 2.80.
Measures
Data from individually and group-administered measures were collected for all student participants to assess the effects of each of the interventions. The measures were selected to capture the decoding, word recognition, and comprehension skills the interventions were designed to address. Pretests were administered during the 2 weeks prior to the onset of intervention to select the sample and assess initial student achievement. Posttest measures were administered within 2 weeks of the end of the intervention. All assessments were counterbalanced so that each student received the assessments after screening in a different order, with all possible orderings used equally in administration.
The assessments were given by trained data collectors who did not know the intervention conditions. Examiners were trained and participated in administration practice sessions prior to each round of testing. Examiners were required to administer each assessment to someone taking the exam with scripted answers and errors. Scoring was checked and, if discrepancies were found, examiners completed a second round of practice to ensure accurate scoring. Interrater reliability was then calculated by taking the number of agreements and dividing by the sum of the number of agreements and disagreements. Data collectors demonstrated at least 90% reliability on the administration and scoring of each measure.
WJ III
The WJ III is an individually administered battery of cognitive and achievement tests. We administered the Letter Word Identification, Word Attack, Oral Comprehension, and Passage Comprehension subtests to assess word reading ability and comprehension. Letter Word Identification assesses the ability to read real words. Word Attack measures the ability to decode nonsense words. Oral comprehension is an oral cloze activity in which students listen to a short audiorecorded passage and supply the missing word. The Passage Comprehension subtest is a cloze measure wherein students silently read sentences and supply missing words. The test begins with simple passages and progresses to more complex passages. Test–retest reliabilities for Grade 4 are reported as .85, .81, .74, and .86 for Letter Word Identification, Word Attack, Oral Comprehension, and Passage Comprehension, respectively.
Gates-MacGinitie Reading Tests
The Gates-MacGinitie (MacGinitie et al., 2006) is a group-administered, norm- referenced reading test for K–adult. We administered the vocabulary and comprehension subtests. Vocabulary presents words in context. The student chooses the correct meaning of the target word. Comprehension provides students with reading passages followed by multiple-choice questions. Questions address facts, inferencing, and drawing conclusions. Test–retest reliabilities were above .85 for both vocabulary and comprehension.
Results
A series of ANCOVA analyses were conducted to examine differences in means between groups at posttest. ANCOVA in the context of a randomized trial design can reduce unexplained variance in outcomes, thereby increasing statistical power (Van Breukelen & Van Dijk, 2007). The pretest scores for the respective dependent variables were used as covariates in the analyses. Effect sizes were also calculated using a procedure by Lipsey and Wilson (2001) to examine gains from pretest to posttest for both groups. Students’ status as LEP was also evaluated as a moderator of treatment.
Pretest
Tables 2 and 3 provide the pretest and posttest means and standard deviations for each of the study groups. The t test for independent samples showed no statistically significant differences between any of the pairs of study groups on pretest measures of word identification, word attack, vocabulary, or comprehension.
Study Group Raw Score Means and Standard Deviations
Note: Gates = Gates-MacGinitie Reading Tests; WJ III = Woodcock-Johnson Psycho-Educational Test Battery-III; Pre = pretest; Post = posttest. Standard deviations in parentheses.
n = 21.
n = 24.
n = 19.
n = 23.
Study Group Scaled Score Means (Gates), Standard Score Means (WJ III), and Standard Deviations
Note: Standard deviations in parentheses. Gates = Gates-MacGinitie Reading Tests; WJ III = Woodcock-Johnson Psycho-Educational Test Battery-III; Pre = pretest; Post = posttest.
n = 21.
n = 24.
n = 19.
n = 23.
Primary Posttest Results
There were no statistically significant differences between study conditions on the WJ III subtests of Word Attack, F(3, 82) = 1.271, p = .290, d = .04–.54; Word Identification, F(3, 82) = .373, p = .773, d = .03–.24; Listening Comprehension, F(3, 82) = 1.478, p = .227, d = .02–.52; or Passage Comprehension, F(3, 82) = .950, p = .421, d = .07–.54. Similarly, no significant differences were noted on the Gates subtests of Vocabulary, F(3, 82) = .270, p = .847, d = .04–.17, or Comprehension, F(3, 81) = 1.987, p = .123, d = .05–.84.
Moderate effect sizes (d = .41–.54) were noted for the word recognition emphasis group on the Word Attack measure when compared with each of the other study groups. However, these effects were not present on the Word Identification measure. A small effect size was noted in favor of the responsive emphasis condition when compared with the comparison group on the Word Identification measure (d = .24). Moderate to large effects (d = .71–.85) were noted for the comparison group on the Gates Reading Comprehension subtest when compared with each of the other study groups; however, these effects were not present on the WJ III Comprehension measure. The word recognition emphasis group did demonstrate small to moderate effects (d = .31–.54) on the WJ III Comprehension measure when compared with each of the other study groups. Mixed effects were noted on the Listening Comprehension measure, with moderate effect sizes found for the word recognition emphasis and the responsive emphasis groups when compared with the comparison group (d = .41–.43). The word recognition emphasis group also demonstrated a moderate effect size over the comprehension emphasis group on the Listening Comprehension measure (d = .52).
Moderator Analysis
LEP status was included as a covariate in the above models to evaluate its potential moderating effect. LEP status was not associated with treatment effects on WJ III Listening Comprehension or Passage Comprehension measures or on the Gates Vocabulary measure. LEP status did predict differences in outcomes on measures of word attack, F(1, 78) = 4.711, p = .033, and word identification, F(1, 78) = 4.572, p = .036, favoring students with LEP. However, the interaction of LEP status and treatment was not significant indicating the LEP main effect did not differ by treatment group.
Discussion
This study examined the relative effects of reading interventions with varying instructional emphases on the reading outcomes of Grade 4 students with reading impairments. The results indicated there were no reliable differences between the study conditions in producing student reading gains. No statistically significant differences were noted in student reading outcomes after participation in an intervention emphasizing comprehension skills and strategies or an intervention emphasizing word recognition skills and strategies. In addition, students receiving the researcher-implemented interventions demonstrated statistically similar reading outcomes to students who continued to receive the school-implemented interventions. Effect size calculations show a trend in favor of the word recognition emphasis group related to word attack and in favor of the comparison group on the group-administered comprehension measure that also served as the selection measure. However, we interpret these practical effects cautiously given that similar trends did not appear on other measures of these constructs and there were no statistically significant findings. Students with LEP performed better than other students on measures of word attack and word recognition in all study conditions.
These results suggest that outcomes for students with reading impairments in the upper elementary grades were not reliably different whether basic skills or higher level skills were emphasized, even when students’ initial levels were taken into account. Both interventions were designed to increase students’ ability to read and understand text. The amount of time spent reading connected text was controlled across the three treatment conditions. In addition, both interventions included instruction in word recognition, vocabulary, and comprehension, with only the explicitness of instruction and the amount of time devoted to either word recognition or comprehension varying.
The school comparison group also did not demonstrate reliable differences from the treatment interventions. In considering the instruction these students received, we note that all students in the study (treatment and comparison) were receiving test preparation skills and practice in their classrooms. In the district, this instruction typically includes practice reading passages and answering questions with instruction in how to use context to determine unknown word meanings, identifying the main points of passages, inferencing, and how to locate answers in text. In addition, all students (treatment and comparison) were instructed in one of CSR strategies, Get the Gist, as part of their classroom instruction. Thus, all students in the study had equal access to instruction in one of the nine strategies of CSR instruction. About half of the students in the comparison group also received additional practice and instruction in reading through school-provided interventions that were aligned with the classroom instruction and focused on test preparation skills and strategies. The findings suggest the instruction implemented in each of the conditions was not reliably different in the outcomes for participating students and no specific study condition benefitted student outcomes above and beyond the instruction provided in the comparison conditions.
Students with LEP did demonstrate accelerated learning on measures of word attack and word recognition in each of the study conditions over non-LEP students. This finding suggests that the students with LEP might have had a better understanding of the sound structure of written English language following the interventions. The lack of differential effects by study condition indicates that the type of intervention was not directly related to the accelerated learning for these students. In fact, reading intervention provided in small groups may provide necessary instruction and practice in reading English text to increase word-reading achievement for students with LEP.
Intervention Intensity
The schools where this study took place have historically implemented research-based reading instruction and have consistently received “recognized” or “exemplary” ratings by the state even though students are from linguistically and culturally diverse backgrounds and low socioeconomic conditions. Thus, the classroom instruction that students with reading difficulties are receiving would typically be considered strong. The study results suggest that none of the interventions employed accelerated student learning on standardized measures of reading above and beyond these typical conditions. Although learning gains from pretest to posttest were noted for all conditions on each measure, it is clear from the outcomes that the participating students were still struggling in the area of reading. For students with reading impairments in the upper elementary grades, more intensive interventions than those provided in this study may be needed to accelerate learning for students who demonstrate persistent reading difficulties within the context of effective classroom instruction. Several researchers have also recently noted that a 1-year intervention with only 30 to 60 min of instruction per session may not be enough to offset several years of reading failure (Kim, Samson, Fitzgerald, & Hartry, 2010; Vaughn, Denton, & Fletcher, 2010).
More individualized interventions, in which instruction is developed based on individual student traits and progress, have historically been considered one way to make interventions more intense for students with severe learning impairments. However, currently there is little empirical work examining individualized work. Although not an individualized intervention, the responsive intervention in this study that attended broadly to student needs in word recognition and comprehension did not demonstrate better outcomes for students. Mathes et al. (2005) also found a responsive intervention was not superior to a standardized intervention even for early struggling readers. Vaughn and colleagues (Vaughn, Wexler, Roberts, et al., in press) recently compared an individualized intervention for middle school students with significant reading impairments to a standardized intervention. No statistically significant differences between the two treatments were found on measures of word recognition, decoding, spelling, fluency, or comprehension. However, in a second study by this research team, it was demonstrated that for Grade 6 students, 3 years of intervention was the amount needed before student reading outcomes were significantly higher than those of a comparison group using school intervention (Vaughn, Wexler, Leroux, et al., in press). Clearly, there are no easy answers for effectively intensifying interventions for students who struggle with reading in the older grades, and continued research in this area is important.
One way the intervention could have been more intensive is to provide more time in instruction. While we had originally hoped to provide 45 min of daily intervention to the students in the research interventions, the school scheduling only allowed for 30-min interventions. In elementary schools, 30-min interventions are a common amount of time for intervention (Wanzek & Cavanaugh, in press), and it can be difficult to schedule longer interventions for all students requiring intervention; however, it may be necessary to have more dedicated time to assist students in making additional growth in reading.
We also did not implement an intervention combining the word recognition and comprehension emphasis. It may be that this combination, providing equal emphasis of instruction in both basic skills and higher level skills, would increase the intensity of the intervention sufficiently for student learning. However, it is unlikely that the combination of these two interventions could be effectively accomplished in a 30-min intervention.
Previous Research on Upper Elementary Interventions
In contrast to the large effect sizes found for interventions focusing on comprehension in previous research with upper elementary students (Wanzek et al., 2010), we found no differential effects for treatment on standardized reading comprehension measures. Effect size calculations in this study were overall small, with moderate effects for the word recognition emphasis group on the WJ III comprehension measure and large effects for the comparison group on the Gates comprehension measure. All of the studies included in the Wanzek et al. (2010) synthesis measured student outcomes on researcher-developed, intervention-specific measures. We measured student comprehension outcomes on standardized measures of broad comprehension abilities in this study. Affecting these broader comprehension outcomes consistently may be more difficult and require different or more intensive interventions than those employed in this study.
The previous synthesis of research for upper elementary students also reported small to medium effect sizes on a variety of outcome measures for studies examining interventions that focused on word recognition (Wanzek et al., 2010). Similar to the previous research, we also found overall small effects. The word recognition focused intervention demonstrated moderate effect sizes on the word attack measure, but none of the effects was statistically significant.
The current study findings also have many similarities to the findings of a reading intervention study with Grade 5 students with reading difficulties (Torgesen et al., 2006). Although the interventions in the Torgesen et al. (2006) study were implemented for longer periods of time (55 min daily) and in smaller groups (1–3 students), the treatment interventions replaced the classroom instruction. Grade 5 students in the study realized significant effects on two of seven reading outcome measures (word attack and sight word efficiency) when compared with students at the school participating in the classroom instruction and school- implemented intervention. The significant effects noted in the Torgesen et al. study produced a small effect size (ES = .18) on word attack. Although we noted a larger effect size on word attack for the word recognition emphasis intervention in the current study (ES = .46), this effect was not seen on the word identification measure and none of the effects were statistically significant. The effect size for sight word efficiency in the Torgesen study was reported as .09.
Kim et al. (2010) also noted similar findings in a study of the multicomponent afterschool reading intervention for upper elementary students when compared with afterschool instruction that was not focused on reading instruction. No significant differences were found between the treatment group and comparison group on any standardized reading measure. Grade 4 students in the afterschool treatment group did outperform the comparison students in oral reading fluency; however, this finding was not replicated at the Grade 5 or 6 levels. The findings from these previous studies, including the current study, suggest that for upper elementary students with reading impairments, significantly more intensive interventions are necessary and may be required for several years throughout secondary grades. However, the current study did note the accelerated benefit for LEP students when receiving reading intervention.
Limitations and Implications
The sample for this study was relatively small and thus had low power for detecting statistically significant findings. However, we have closely examined the effect sizes, standard score gains, and individual student growth, and are confident that the results are fairly consistent. Although individual students would likely be documented as responders to the interventions, and all mean standard scores were increasing, there were no statistically significant differences in posttest scores between the interventions and, overall, none of the interventions was powerful enough to place the majority of students into the average range on the comprehension measure. Although some promising effect sizes were noted, the findings were not consistent across measures of similar constructs and should be interpreted cautiously, given the nonsignificant results. Proximal measures aligned with the intervention might have also provided some information to indicate whether students were achieving some gains in the intervention such that with additional intervention the following school year, they could have continued to a level positively affecting broad reading outcomes. Future research in the intensity of interventions necessary for students beyond Grade 3 could help to shed light on these issues. Importantly, students with LEP had higher scores in the interventions than their peers indicating the importance of reading intervention for these students.
Further research is needed to examine effective interventions for students with reading impairments in the upper elementary grades. Thus far, research, including this study, has suggested that small effects can be obtained for students participating in interventions focused on word recognition and no effects have been documented for comprehension instruction on standardized comprehension measures. Few studies incorporating multiple components, word recognition, and comprehension have been examined for students with reading difficulties in the upper elementary grades (Wanzek et al., 2010). We did not implement an intervention combining the emphases in word recognition and comprehension that were implemented in this study, but that may be one way to make the interventions more powerful. Future research needs to examine the interventions incorporating a focus on basic skills as well as explicit instruction in higher level skills for upper elementary students.
We believe that the findings from this study provide opportunities to consider the impact of treatments on students with persistent reading difficulties who have received relatively strong classroom reading instruction. These students, often considered Tier 3 or Tier 4 students in a Response to Intervention model, are among the students about whom we have the least information in regards to how to effectively increase their reading success. The lack of clear information for significantly accelerating student learning with current methods in several well-designed studies cannot be ignored. Further research examining long-term and intensive treatments with this population is needed.
Footnotes
The content is solely the responsibility of the authors and does not necessarily represent the official views of The Meadows Foundation.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a grant from The Meadows Foundation.