Supplemental Mathematics Intervention

Teachers

Recruitment of elementary school special education teachers in a large urban district in the Southwestern United States consisted of several inclusion criteria. Teachers had to be (a) teaching mathematics interventions as part of daily instructional responsibilities (i.e., Tier 2 instruction within a Response to Intervention [RtI] model); (b) teaching mathematics interventions to third-, fourth-, and/or fifth-grade students with Individualized Education Program (IEP) goals in mathematics (LD in particular); (c) experienced with at least 3 years teaching mathematics intervention lessons; and (d) currently teaching students in intervention settings who had IEP goals in numeration and/or basic operations. Recruitment of teachers was accomplished by (a) identifying possible school sites supplied by the district as willing and interested to participate in the research study, (b) arranging school site meetings with special education teachers who met the inclusion criteria for the study, (c) conducting meetings with teachers to establish interest and ability to participate, and (d) obtaining consent from teachers and their students.

Researchers selected a sample of 10 special education teachers from three elementary schools. A majority of the teachers were female (n = 9) and White (n = 7). One teacher was Black and 2 teachers were Hispanic (1 being male). At the time of the study, all teachers held bachelor’s degrees, 2 teachers held master’s degrees or master’s degrees plus additional credit hours. All teachers had at least 5 years of teaching experience, 4 teachers had 6 to 10 years of teaching experience, and 2 teachers had 11 to 20 years of teaching experience. Teaching certification included special education (n = 3), elementary education (n = 1), or both (n = 6). Four teachers taught the third-grade intervention lessons (i.e., Base Ten Numeration, see below for a description) and 6 teachers taught the fourth-grade intervention lessons (i.e., Multiplication/Division Strategies).

Students

Twenty-three students constituted the student sample and were (a) in the third, fourth, or fifth grade; (b) identified as LD on their IEP; (c) had IEP goals in the areas targeted by the intervention (i.e., numeration and basic operations); and (d) scored below 50% on a curriculum-based measure (CBM). A score below 50% on the CBMs substantiated students’ existing IEP goals made by the district in the targeted area (i.e., place value and basic operations) and thus their need for further instruction. The district identified students as LD using the duel discrepancy approach (difference between achievement and IQ). One student was identified as having intellectual disabilities (ID); the student was included in the study because he or she was a natural part of the instructional group utilized in the school setting. Nine students completed the Base Ten Numeration module, and 14 students completed the Multiplication/Division Strategies module.

Lesson content

A Tier 2 intervention served as the supplemental mathematics intervention in this study. Elementary Students and Teachers Algebra-Readiness for Grades 3 and 4 (ESTAR) was developed as a latter elementary school version of effective, foundational mathematics intervention (D. P. Bryant et al., 2008). The lessons are research based, and the topics selected are considered as foundational for success in higher mathematics coursework (e.g., number and operations concepts and procedures; National Mathematics Advisory Panel, 2008). The Base Ten Numeration module focused on developing notions of and proficiency with the base-10 system (i.e., quantity and number values). In each lesson, students used multiple representations of multi-digit numbers while working through a sequence of activities that included (a) decomposing and composing number modules, (b) counting with varying grouping modules and discussing relationships between them, and (c) comparing and ordering. Fluency activities, including number recognition/naming and writing numbers for stated quantities, were also utilized throughout all lessons in the module.

The Multiplication/Division Strategies module focused on developing procedural fluency of basic multiplication and division facts. In each lesson, students utilized multiple representations alongside problem-solving strategies to develop an understanding of how to solve facts quickly and efficiently. Students used number relationships, composition, and decomposition to solve (a) two digits by one-digit, (b) two digits by two-digit, and (c) three digits by one-digit multiplication and corresponding division problems. The sequence of fact strategies was planned according to difficulty and number strategies. Fluency problems were utilized throughout all lessons in the module.

Lesson components

Each lesson included instructional design elements consistent with recommendations for mathematics instruction for students with LD (Gersten et al., 2009) and included the following components: (a) cumulative review, (b) engage prior knowledge, (c) modeled practice, (d) guided practice, and (e) independent practice. Instructional time for each component varied: 3 min each for cumulative review and engagement of prior knowledge with modeled practice, guided practice, and independent practice generally lasting 10 min, 8 min, and 6 min, respectively. Intervention duration was 30 min 3 days per week.

Delivery of instruction

Each supplemental instruction lesson was scripted. Design of lesson scripts was based on research-supported features of mathematics intervention for struggling learners outlined in the review of literature (i.e., teacher modeling of strategies to be used by students to learn new material, student restatement of teacher-modeled ideas during practice, purposeful review of previously mastered content, and active engagement through providing students with multiple opportunities for students to respond). The following paragraphs explicate varying teacher and student actions included in the specific lesson components described earlier.

First, in modeled practice, explicit, systematic instructional procedures were included in the script (Gersten et al., 2009). Procedures included an emphasis on clear, transparent modeling of strategies for solving problems, checking for understanding, individual and choral response, and immediate correction of erroneous responses using suggested error correction strategies within the lesson. For instance, when teaching multiplication of a two-digit number by factors of 9, the teacher may say, “When multiplying a number with two digits by a factor of 9, we use the strategy ‘10 minus the factor.’ What strategy? Get ready . . . .” If quick and accurate responses are given, the teacher praises and continues with demonstrating the steps of the strategy or how to use varying levels of representation to model the procedural steps (Butler et al., 2003). Levels of representation were grounded in tactile materials/representations when concepts were newly introduced; tactile representations were replaced with figurative and symbolic representations as concepts were repetitively modeled and explained.

Second, during guided practice, the script directed teachers to work several examples with students using scaffolded levels of questions to support the transition of strategy and procedure usage to solve problems from the teacher to the student (Coyne et al., 2011). The script began with high-level support questions. If students responded to teacher questions with a low degree of error, the script transitions to medium-level support questions. When utilizing medium-level support questions, the teacher continues to look for students’ correct and quick recitation of taught procedures and strategies. The difference was that questions are used in fewer numbers in an effort to relinquish some control or authority for mathematical “doing” onto students. Finally, the teacher shifts to low-level support questions (fewer questions allowing elaboration of steps or procedures on the part of the student) as students practice the taught procedures and strategies with little support from the teacher. Activities were also introduced at this point to reinforce learned strategies or procedures to solve problems.

Third, in independent practice, the script directed the teachers to eliminate questioning and supports altogether. Students were given several problems that were modeled during modeled practice and practiced together during guided practice. To consider the lesson successful, students needed to correctly solve 75% to 80% of problems (dependent on amount of problems given) independently.

Screening

To confirm students’ need for intervention, students’ initial performance in the targeted areas was assessed through a distal CBM for each intervention module. The CBMs were created from the Independent Practice sheets. The CBMs contained 40 items, 2 items from each lesson’s Independent Practice. Items included situated problems and abstract problems. Researchers calculated split half reliability of the measures in a pilot study. Results of the reliability analyses generated an alpha coefficient of .87 for the Base Ten Numeration module and .91 for the Strategies module (Nunnally & Bernstein, 1994). Content validity was established because the items on the screening CBM were drawn from the independent practice items for each module.

Fidelity of implementation

Researchers utilized lesson observation forms to record the extent to which each teacher adhered to the instructional script for each lesson observed. Construction of the observation forms began from copies of each lesson script used by teachers in both the Base Ten Numeration module and the Multiplication/Division Strategies module. Researchers inserted a column into the left and right margins of each scripted page of each lesson. The researchers recorded whether teachers were (a) implementing each scripted phrase or directive as they taught the intervention (i.e., a checkmark in the left margin of the observation form indicating each statement/directive was completed) or (b) making changes to the script relative to each lesson component (i.e., modeled practice; guided practice) as they taught it (i.e., a checkmark in the right hand margin of the observation form indicated a change, and the change was noted). Eighteen lessons were observed for each teacher in third grade, and 15 lessons were observed for each teacher in fourth grade.

Inter-observer agreement (IOA) was established through an examination of 30% of all observation forms completed over the course of the study. The number of times observers agreed (i.e., observers agreed that a lesson element was performed without a change or observers agreed that a lesson element was performed with a change) was 103 (M = 145). Disagreements were 21. We determined inter-rater reliability using Cohen’s (1960) kappa. The statistic produces a possible range of agreement between −1 and +1 and is a strong gauge of observed agreement between coders as it corrects for agreement that would be expected by chance. The analysis (i.e., agreements, adjusted for chance divided by agreements + disagreements, adjusted for chance) yielded a kappa of 0.87, suggesting substantial agreement among coders (Hallgren, 2012).

Pedagogy alteration form

The second data source was information gathered from teachers concerning how lessons were altered. Teachers were asked to record their alterations by writing any alterations they made to the lessons in the margins of the lesson booklets. Alterations were broadly defined to teachers as any change they made to scripted phrases, activities, tasks, or materials called for in the lessons in each lesson component.

Focus group

A 2-hr focus group was held at the conclusion of the study. The focus group was utilized to obtain a holistic account of the nature of alterations made during instruction and teachers’ perspectives on why the changes were needed during instruction. Ten questions were developed at the onset of the study to uncover perceptions of intensification of instruction and perceived need (or the lack thereof) for altering lesson components grounded the focus group discussion. Each question contained 1 to 3 probing questions; a standard protocol was used to guide questioning. The focus group discussion was audiotaped; anecdotal notes were also gathered to triangulate data collection (Krueger, 2009).

Student consent, assent, and CBM administration

Approval for conducting the study was obtained through the Institutional Review Board and the research division of the school district. All 10 teachers identified for the study signed and returned the consent forms. Identified students were then given parental consent forms for study participation. Student assent was collected through a signature on the bottom of the consent form. All students who met the selection criteria returned consent forms. Next, all identified students completed the researcher-developed CBM screener for either the Base Ten Numeration or the Multiplication/Division Strategies module. CBMs were given to students as a group. Students were told they were going to complete questions about base ten or multiplication/division strategies and to do their best; no other direction was provided. Word problems contained in the tests were read out loud to students who requested assistance. All students scored below 50% on the CBMs.

Teacher training and lesson implementation

Prior to lesson implementation, teachers received 3 hr of training on all lessons in the modules. The training included an overview of the lesson components and content areas as well as a detailed explanation and modeling of each lesson element. The project coordinator and a second member of the research team acted as the teacher for a selected lesson from each grade level; teachers acted as students as the content was delivered. In addition, teachers were given time to examine the lessons in groups and practice delivering lesson components. Teachers were instructed to deliver all lesson components as presented but to make alterations when warranted by student needs. Teachers completed feedback forms at the conclusion of the professional development. Scores from the feedback forms reflected a high degree of satisfaction with the content and structure of the training. Instruction commenced over a 7-week period beginning in February and concluding in March. The participating students received instruction within a special education resource classroom during their assigned mathematics time as defined in students’ IEP. The lessons were taught in groups of two to three students at a semi-circular table; there were no other students present in the room. Students were instructed with the module lessons three times per week for 30 min per session. The 3 days varied across schools as a result of individual school schedules; district standardized testing schedules, and available instructional time per school site.

Lesson implementation and alteration data

Lesson implementation data were collected through researchers’ observations of lessons (i.e., fidelity of implementation lesson observation forms) and teacher’s own descriptions of alterations (i.e., pedagogy alteration forms). Prior to observing teachers in instruction, the first author trained the research team on the use of the observation forms used to collect data for the study. Researchers met for a period of 3 hr to discuss the lesson components and how to use the observation forms. Two researchers were present and observed each teacher as they implemented the modules over the 7-week intervention period. At the conclusion of the study, researchers also collected teachers’ notations on the lesson booklets of the alterations they made as they implemented the intervention modules.

Focus group data

Prior to the focus group, all researchers reviewed the processes of interview skills and content analysis procedures to increase the analytic consistency among the researchers with another researcher to assure reliability and validity within the later created process audit trail (Krueger, 2009). One researcher was assigned the role of facilitator. The facilitator posed questions to the teachers, encouraged discussion, and probed for elaboration of ideas brought to bear during discussion. The other researchers were assigned the roles of anecdotal note takers and took detailed accounts of responses to facilitator questions and who was responding, and directions of discourse (flow of conversations). This was done to ensure an audit trail and consistency of data collected (Glaser & Strauss, 1967).

At the outset of the focus group, permission from the teachers to be audiotaped was obtained. An explanation of each participating researcher’s role and purpose for the focus group was provided. Participants were guided using a semi-structured interview format (Glaser & Strauss, 1967). The questions related to overall perceptions of the lessons, their implementation, teachers’ alterations to the intervention lessons, and the factors that, from the perceptions of the teachers, influenced the alterations. Questioning also sought to uncover the teachers’ perceptions of intensification related to alteration of varying lesson components. Member checks were completed for each question before asking subsequent questions. The facilitator concluded the focus group with a summary statement, requested additions from the group as a further means of member checking, and ceased audiotaping.

Research Question 1: Extent of lesson alterations

Descriptive statistics were calculated to obtain an overall gauge of how much change was or was not being made to lessons in each lesson component for each module. To do this, researchers gathered all lesson observation forms conducted across all teachers who taught that module. Researchers focused on a particular lesson component (e.g., modeled practice) and counted how many instructional statements and/or directions in that component were delivered with changes and divided that amount by the total number of instructional statements and/or directions in that lesson component (e.g., 13 statements/directives changed within modeled practice out of 15 total statements/directives within modeled practice yields a change percentage of 87%). Researchers repeated this process for all remaining observation forms for that particular component. Finally, an average was taken across all lessons for that component to obtain an overall percentage reflective of the extent to which teachers, in their instruction, were altering various lesson components within each instructional module.

Research Question 2: Nature of lesson alterations

Researchers utilized a constant comparison method (Glaser & Strauss, 1967) and classical content analysis (Leech & Onwuegbuzie, 2007) to delineate the nature and number of observed lesson alterations. Data included the completed observation forms from researchers and the teachers’ completed lesson booklets with self-reported alterations. First, researchers read through all completed observation forms and teacher’s lesson booklets as a team. Next, the data were chunked into smaller, more meaningful parts. Researchers negotiated a mutual set of codes, with examples for each, to assure content validity of the analysis. Codes were defined as pedagogy alterations, materials alterations, and task alterations (see “Results” section for further details on the codes). After independently coding sub-samples of data sets using the defined categories, the researchers conferred to compare responses. Interrater reliability, agreements / (agreements + disagreements), indicated a range of agreement from 87.9% to 91.4% (M = 89.8%). Next, researchers used classical content analysis to discern how many times certain types of alterations were used by teachers across the essential elements in each lesson observed. This descriptive information about the data was complementary to the constant comparative analysis. Researchers used the codes delineated during the constant comparison analysis to count how many occurrences comprised each code in the data set and in which lesson component these codes were observed.

Research Question 3: Perceived need for alterations

Researchers used constant comparison methods (Glaser & Strauss, 1967) using transcription from the focus group audiotapes. Categories for analysis were generated and defined by the researchers who independently examined the data. For each issue or question, the responses were reviewed for common ideas and themes, which were used to develop an initial list of categories and analyzed using guidelines suggested by Grbich (2007) for data analysis and reduction. Researchers met and negotiated a mutual set of categories, with examples for each, to assure content validity of the analysis. Next, researchers developed data summaries. Using matrices, the researchers summarized key findings for each of the topics/questions generated by the researchers. Finally, conclusions from the data analyses were developed and verified. Conclusions were drawn over time and reported as they were found to be explicit and grounded (Glaser & Strauss, 1967). Verification was conducted through the group process and through a final check of inter-rater reliability to ensure consensus (range = 83.0%−89.4%; M = 86.2%).

Base Ten Numeration module

For cumulative review, the overall change percentage was 12%, whereas the change percentage for fluency was 4%. The percentage of changes was highest (21%) for modeled practice. Changes for guided practice and independent practice were 15% and 9%, respectively. Overall, the percentage of changes data indicated that the cumulative review, fluency, and independent practice components received a relatively low degree of teacher alterations, while guided practice and modeled practice were the lesson components teachers changed the most.

Multiplication/division strategies module

For cumulative review, the overall change percentage was 9%. For fluency, the change percentage was 6%. Interestingly the change percentage for modeled practice was high at 43%, whereas for guided practice and independent practice change, percentages were 27% and 12%, respectively. Thus, like the Base Ten Numeration module, cumulative review, fluency, and independent practice were slightly altered compared with guided practice and modeled practice, which received the most changes.

Scripted lessons and tasks

The first theme related to special educators’ appraisals of the scripted lessons and tasks within the lessons, which are typical for a standard protocol for supplemental interventions (D. P. Bryant et al., 2011; L. S. Fuchs, Fuchs, et al., 2008). Nineteen teacher statements focused on alterations to (a) bridge to students’ prior and informal knowledge, (b) differentiate among varying levels of students’ understanding or missing skills, and (c) provide additional opportunities for practice. Six statements coded within this theme illustrated teachers making changes to the instructional script when they felt it was insufficient for relating to notions students did possess. One teacher explained,

. . . with respect to having to do things a certain way—that might not always connect to what the student could do or wanted to do. Lots of assumptions of ways kids do things, like strategies students used that were correct but different than what was modeled due to how they were thinking about the math.

Seven statements illuminated some of the teachers’ attempts to differentiate lesson content to address specific areas of student need (e.g., address certain “missing” skills or misconceptions), which is important for students with LD who demonstrate mathematics difficulties.

Finally, six statements elucidated some of the teachers’ propensity to intensify the lessons by incorporating additional opportunities for practice. The insertion of additional practice opportunities usually involved an effort to ensure students’ mastery of the content. One teacher explained,

. . . And so I’ve found that I’ve had to . . . add more examples for me to feel confident to move on and to demonstrate. So it’s like I just make sure that it’s . . . at least an 80 or above at that point, before we move on to practice. Some days I go, oh we’ll be done with this in 30 minutes and it’s a 3-day lesson. I mean it’s just, so I just never know what it’s going to be, but I’ll stay and demonstrate for 2 days if I have to until I feel confident that they can go on. Because I’m not trying to “finish,” I’m trying to teach them math.

Connections

The second theme revealed a perceived need to alter the modeled practice and guided practice components of the lessons to intensify the interventions by providing additional connections. The connections teachers spoke of in the 23 statements that fell under this theme referred to (a) decontextualized tasks or lesson scripts that did not connect to students’ daily life experiences; (b) lessons delivered in a largely procedural manner that did not seem to connect to a larger concept; and (c) the addition of visual, kinesthetic, or verbal “cues” to assist students in associating or remembering a concept. Seven of the statements that fell under this theme referenced adding or altering the context of modeled practice and guided practice scripts and tasks to make them more familiar to students’ daily lives:

You know like if you’re going to talk about division maybe if they would just have prior to going into that, some real life examples of where you use division . . . And I think that you know the story problems are good and the application part, but you’ve already taught all the stuff and now you’re doing that. And so I sort of added a lot of those prior to trying to teach it. You know trying to get them to connect [the concept].

Five other statements spoke of teachers needing to scaffold the lesson scripts to bridge procedural aspects of mathematics skills delivered in the lessons to a more conceptual base:

Yeah I guess I’m not saying the procedures aren’t right, but I guess they, they’re getting so caught up in the procedures of where the numbers go that they’re not connecting to what the procedures are trying to teach them . . . . They’re not getting it. They’re not getting the big picture.

Teachers did not perceive students’ misuse of procedural steps as a deficit in knowledge on the part of the student; instead, teachers commented the lessons did not present procedures in a manner that would promote adaptive procedural expertise. One teacher commented,

I just felt that we got caught up in a lot of the procedure of doing something . . . there were too many procedures, there were too many steps [in that] we completely lost the focus of what we were trying to learn: the distributive property.

Other statements that fell into this theme spoke of connections as visual, verbal, or gestures-based “cues” to aid students’ recall of the content; 11 statements defined “connections” in this way:

So it’s incorporating those different pieces and maybe relying a little less on the language—just my talking—versus them being able to know they can go to that visual piece. For example, doubling is one concept that just continues to confound them. But with a visual picture of things doubling, or the manipulative they’re doubling, it reconnects that concept.

Lesson delivery changes

The third theme illustrates lesson delivery changes. The changes teachers spoke of in the seven statements coded within this theme referred to (a) changing the delivery to involve more of a student-centered pedagogy (two statements) and, relatedly, (b) increasing opportunities for students to explain and justify their thinking (four statements). Some teachers viewed the lesson delivery as too teacher-centered for the students they instructed, with an insufficient emphasis given to students’ role in their own learning of mathematics.

In the demonstrate part . . . even though I’m showing them what to do, I find myself trying to get different response methods from them because it’s like they rely entirely on me and not on their own minds. So I changed it to asking . . . can you explain that back to me or is there another way you were thinking about it. Somehow, there has got to be more; activity . . . thinking . . . from them.

Other teachers identified mathematical reasoning and justification as a missing student activity.

And the question that I’ve been saying to them is . . . if I know that’s wrong, I go “well prove it” and ask why it’s right. And I have them teach it back to me how they got that answer and then I say to the other person, “do you agree with that?” So the whole thing, the premise behind when they teach another child, they learn it better, then it becomes real to them.