The Effects of an Inference-Making Strategy Taught With and Without Goal Setting

Abstract

This quasi-experimental study examined the effects of a strategy for making text-dependent inferences—with and without embedded self-regulation skills—on the reading comprehension of 24 middle-grade students with disabilities. Classes were randomly assigned to receive the inference intervention only (IO), inference + individual goal setting (IIG), or inference + group goal setting (IGG). The Kruskal–Wallis one-way analysis of variance revealed no significant differences between groups on overall reading comprehension performance, but students in the IGG group significantly outperformed the IO and IIG groups on evaluation items, χ²(2, N = 24) = 13.18, p = .001. Paired samples t tests indicated all groups significantly improved their comprehension performance from pre- to posttest, IO: t(8) = 2.76, p < .05; IIG: t(6) = 3.97, p < .01; IGG: t(7) = 4.35, p < .01. The IGG group wrote significantly more valid inferences in Lessons 3 to 7 than the IO or IIG groups; χ²(2) ranged from 7.26, p < .05, to 16.16, p < .001.

Keywords

inferences goal setting adolescents reading comprehension

After the development of basic decoding and fluency skills, continued progress in reading comprehension relies on the ability to make inferences that connect information across the text and with background knowledge or that fill gaps that would otherwise erode the coherence of the text (Cain, Oakhill, Barnes, & Bryant, 2001; van den Broek, 1997). In fact, a study of inference making found it uniquely predicted reading comprehension in 10- and 11-year-olds even after controlling for verbal intelligence, vocabulary, and story structure knowledge (Oakhill & Cain, 2012). Given the need for a threshold level of basic skills and vocabulary knowledge, researchers generally seem to agree that inference making is best taught in the middle grades (cf., L. Baker, 2002). This is also a time when the passages students read become longer and more complex—a circumstance more likely to enable the formulation of inferences with greater specificity and wider variety (Gygax, Garnham, & Oakhill, 2004; van den Broek, 1997).

The increased expectation for inference making is reflected in the progression of text and question types on the National Assessment of Educational Progress (NAEP) Reading Framework (National Assessment Governing Board & U.S. Department of Education, 2012). Across the tested grades 4, 8, and 12, the NAEP texts become longer, include more sophisticated elements of various genres, and are assessed with items targeting interpretation and evaluation more so than recall. In addition, students are required to answer more short (i.e., requiring 1–2 sentences) or extended (i.e., requiring 1–2 paragraphs) constructed response items in addition to multiple-choice items. Unfortunately, students with reading disabilities are likely to struggle with inference making and, subsequently, demonstrate difficulties with reading comprehension (Kendeou, van den Broek, Helder, & Karlsson, 2014). This is because inference ability is multifaceted and requires the orchestration of skills that pose problems for those with disabilities.

To infer information, the reader must possess sufficient background knowledge of the words and concepts in the text, efficiently allocate attentional resources to relevant information, hold information from earlier parts of the text in memory to be integrated with later information, and monitor comprehension to detect and resolve inconsistencies as they arise (cf., Kendeou et al., 2014; Kispal, 2008). In addition, readers need to make local inferences within and across sentences of a single paragraph as well as global inferences across an entire text (Graesser, Singer, & Trabasso, 1994; Gygax et al., 2004). Research with middle-grade students has identified that those with reading difficulties struggle even with local inferences (Barth, Barnes, Francis, Vaughn, & York, 2015). But as the distance between the pieces of information that must be related increases, students have exhibited still greater difficulty (Johnston, Barnes, & Desrochers, 2008). Although it is believed to be a malleable skill, more is understood about the dimensionality and contributions of inference making than the strategies that might improve it (e.g., Cain et al., 2001; Graesser et al., 1994; Oakhill & Cain, 2012). As noted by Elbro and Buch-Iverson (2013), many intervention studies have been multicomponent in nature, limiting the ability to disentangle the impact of inference instruction on students’ reading outcomes.

Inference-Making Interventions With Middle-Grade Students

A search of the extant literature revealed four intervention studies specific to inference making with students in the middle grades. Two of these utilized a single paragraph (Elbro & Buch-Iverson, 2013; Holmes, 1985), which would allow for making only local inferences. The other two used short passages of 100 to 400 words (Fritschmann, Deshler, & Schumaker, 2007) or a one-page, multiple paragraph passage (McMaster et al., 2012), which potentially could target both local and global inferences. Students in all four studies received prepared questions to prompt specific inferences, but each study exclusively used fictional narratives (Fritschmann et al., 2007; McMaster et al., 2012) or expository passages (Elbro & Buch-Iverson, 2013; Holmes, 1985). The researchers reported positive effects for the students participating in the interventions—including those with disabilities (Fritschmann et al., 2007), in remedial reading classes (Holmes, 1985), in general education (Elbro & Buch-Iverson, 2013), and at a range of ability levels (McMaster et al., 2012).

The inference-making intervention reported here differed from the previous studies in the following ways. First, students read both fictional and informational narrative passages that were multiple pages in length so that a number and variety of inferences could be made. To support connecting information across the text, participants were taught to use a graphic organizer for recording information stated in the text that could become the basis for an inference. This organizer functioned as an external memory store to overcome the limitations of text memory noted for students with reading difficulties (Hua & Keenan, 2014). Students also could choose from a list of inference stems rather than receiving questions to prompt particular inferences. The stems allowed for local and global inferences (see the “Method” section). Finally, all participants received the inference-making instruction, but the comparison groups included individual or group goal setting to examine the potential benefits of self-regulation.

Goal Setting

Teaching students with disabilities to monitor and regulate their behavior has been effective at improving their access to general education and their post-secondary transition (Harris, Friedlander, Saddler, Frizzelle, & Graham, 2005; Shogren, Palmer, Wehmeyer, Williams-Diehm, & Little, 2012; Test et al., 2009). A synthesis of self-regulation interventions found effects were strongest when goal setting was used to improve productivity and math skills (Konrad, Fowler, Walker, Test, & Wood, 2007). The authors argued for combining goal setting with direct instruction in academic skills because the self-regulation behaviors can only augment, not replace, what students need to know and do to be successful in academic courses.

This can create tension for classroom teachers who must balance the time needed to teach content and skills with the time needed to improve students’ self-regulation. In a national survey, middle and high school teachers responded that time constraints were the leading cause of omitting self-regulation instruction (Wehmeyer, Agran, & Hughes, 2000). In addition, elementary and middle school teachers have reported not having good knowledge of how to teach or embed such skills within academic instruction (Stang, Carter, Lane, & Pierson, 2009).

Not developing self-regulation skills can have deleterious effects on adolescents. Such students are less likely to be exposed to the general education curriculum and standards (Shogren et al., 2012) and more likely to engage in delinquent behavior (Thompson, Whitmore, Raymond, & Crowley, 2006). In fact, as many as 90% (Drerup, Croysdale, & Hoffman, 2008) of juvenile offenders are believed to meet criteria for one or more mental health disorders that are linked to or further compromise the brain’s self-regulatory systems (Thayer, Hansen, Saus-Rose, & Johnsen, 2009).

Aspects of self-regulation, such as goal setting, were found to predict the long-term success of individuals with learning disabilities (LD; Raskind, Goldberg, Higgins, & Herman, 1999). Moreover, there is some evidence that including goal setting within literacy interventions has improved the performance of middle-grade students in the short term. For example, involving sixth- and seventh-grade students with LD in setting goals for their learning was associated with stronger improvements in reading fluency as compared with peers who did not set goals (Swain, 2005). Similarly, a meta-analysis found the average weighted effect of goal setting in writing interventions was moderate (0.57) across four studies involving students in Grades 4 to 8 (Gillespie & Graham, 2014). The authors concluded that “the positive impact of goal setting . . . suggests students with LD possess greater capabilities for carrying out writing processes than they apply spontaneously” (Gillespie & Graham, 2014, p. 469). Whether this might be true of adolescents’ inference abilities while reading is not known.

Purpose and Research Questions

Previous studies have explored teaching students to make inferences on short paragraphs or passages and with the assistance of specific questions (Elbro & Buch-Iverson, 2013; Fritschmann et al., 2007; Holmes, 1985; McMaster et al., 2012), but little is known about the effectiveness of explicit inference instruction with longer passages and more generalized supports. Therefore, this study sought to contribute to the sparse literature on teaching adolescents with reading difficulties to make inferences while reading fictional and informational narratives. The following question guided the research:

Research Question 1: To what extent does an inference-making intervention improve the abilities of middle-grade students with reading difficulties to infer information in fictional and informational narratives (i.e., productive tasks) and to respond correctly to literal, inference, and evaluation multiple-choice test questions (i.e., receptive tasks)?

The present work also examined the effectiveness of adding a self-regulation component. Previous studies of embedded self-regulation have overwhelmingly utilized single case designs and examined outcomes related to school and classroom behaviors with academic improvement serving as a distal outcome (Konrad et al., 2007). Therefore, the study was designed to address the second research question:

Research Question 2: What are the relative effects of combining an inference-making intervention with individual or group goal setting on the reading comprehension performance—using both productive and receptive tasks—of middle-grade students with reading difficulties?

Method

Participants were drawn from two different schools in the southeastern United States that were more than 70 miles apart. School A was a Grade 6 to 8 public middle school, and School B was a juvenile correctional facility for children between the ages of 9 and 14 (see Table 1 for participant characteristics). These sites were recruited for participation because they had dedicated reading or learning strategies classes as a supplement to students’ general education English language arts (ELA) class. Furthermore, juvenile offenders were included in acknowledgment that the students only differ by the setting in which they are being taught. That is, juvenile offenders move between public and correctional schools and experience the same challenges with reading and academic achievement (Wexler, Reed, & Sturges, 2015).

Table 1.

Participant Characteristics.

Participant characteristic	IO (n = 9)	IIG (n = 7)	IGG (n = 8)	Total (N = 24)
Gender
Male	5	5	8	18
Female	4	2	0	6
Grade
5	0	0	1	1
6	9	7	1	17
7	0	0	4	4
8	0	0	2	2
Age (years, months)	12, 4	12, 6	13, 2	12, 4
Disability
ASD	1	0	0	1
ED	0	0	2	2
LI	2	1	0	3
OHI	1	0	0	1
SLD	5	6	1	12
Race/ethnicity
African American	2	1	4	7
White	7	6	4	17
Free/reduced-price lunch	1	2	8	11

Note. IO = inference instruction only; IIG = inference + individual goal setting; IGG = inference + group goal setting; ASD = autism spectrum disorders; ED = emotional disturbance; LI = language impairment; OHI = other health impairment; SLD = specific learning disability.

Participants

Students from School A were all identified with a disability, primarily LD, and were enrolled in a learning strategies class taught by the same special educator. Of the 16 students, 100% agreed to participate in the study. They spent the remainder of their school day in general education classes, including a 55-min general education ELA class. In addition, nine students received a supplemental reading intervention. This was a computer-based reading practice program offering activities aligned to the tested state standards, but it did not deliver direct instruction in reading skills. Two different teachers taught the ELA classes, and one of those teachers facilitated the reading classes.

Students in School B were committed to the juvenile justice facility for a minimum of 6 to 9 months. Of the nine students in the class, eight (i.e., 89%) consented to participate. All were considered to be performing below grade level based on their school achievement and performance on reading assessments administered by the facility on intake, but not all were identified with a disability. It should be noted that under-identification of disabilities has been a problem among juvenile offenders due to high mobility, a lack of proper evaluation in the facilities, and caregivers’ reluctance to acknowledge the issues (Snyder & Sickmund, 2006; Watson, Kelly, & Vidalon, 2009). Students in School B had a 40-min ELA class and a 45-min reading class. These classes met 4 days per week and were taught by the same teacher. The curriculum followed a state adopted textbook series and included reading trade books.

Intervention Teachers

The distance between the schools necessitated that the second author (a graduate assistant with 2 years’ experience working in special education classrooms) served as the interventionist in School A, and the first author delivered the intervention in School B. The typical classroom teachers were occasionally present during the sessions but were not involved in delivering any instruction. In School B, two security guards were required to be present. They enforced the behavior plan when necessary but were not otherwise involved in the sessions.

Measures

The interventionists administered all measures during students’ usual class time. Pre- and posttests were used to establish group equivalency prior to the start of the intervention and to determine within-group gains and between-group differences after the conclusion of the intervention. Student inferences were monitored each session. All documents were scored by two raters, one of whom was not involved in the intervention and was blind to condition.

Pre-/posttest of reading ability

The easyCBM multiple-choice reading comprehension measure (Alonzo, Tindal, Ulmer, & Glasgow, 2006) was used to determine students’ overall reading ability as well as their disaggregated inference and evaluation performance. Students read one narrative easyCBM passage at the sixth-grade level for pretesting and a different one for posttesting to control for practice effects. The sixth-grade easyCBM was selected because 23 of the 24 participants were in Grade 6 or higher, and the one fifth-grade student was the same chronological age as the younger sixth-grade students. Each passage is followed by 20 multiple-choice questions. Of these questions, seven are identified as literal, seven as inference items (also referred to as low-level inferences), and six as evaluation items (also referred to as high-level inferences). For the sixth-grade easyCBM measures, developers reported a Cronbach’s α range of .63 to .67 and split-half reliability of .47 to .52 (Sáez et al., 2010).

Inference strategy

Students’ work from each session was evaluated for the number of attempted inferences and the number of inferences made that were connected to information stated in the text. For the groups with embedded goal setting, scores on the intervention strategy were used to determine whether students met their goals. Inter-rater reliability among the independent scorers was 99% overall with a range of 97% to 100% for each of the three groups.

Intervention Materials

Although students from both schools were expected to make inferences to complete assignments in their ELA and reading courses, the teachers reported they did not teach a specific strategy for doing so. This was confirmed by reviewing the curricular materials used for typical practice as well as observing each ELA class twice and each reading class once. Three fictional narratives and three informational narratives, each 3 to 5 pages in length, were selected from a sixth-grade anthology. The research team read the selections and generated sample inferences to ensure a minimum of five inferences could be made per passage.

From the sample inferences and the extant literature (e.g., Fritschmann et al., 2007), researchers created a list of 12 stems that students could use to support their work such as, I think the reason _____ happened was because _____. In addition, a graphic organizer (Elbro & Buch-Iverson, 2013; McMackin & Witherell, 2005) was adapted to provide a structured means for students to record stated information from the passage and the inferences they generated from that information (see Figure 1). The stems and graphic organizer were used as scaffolds for the students with learning difficulties, who are known to struggle with written tasks (S. K. Baker, Chard, Ketterlin-Geller, Apichatabutra, & Doabler, 2009).

Figure 1.

Inference-making graphic organizer.

To facilitate goal setting in two of the three intervention groups, the researchers created a form (see Figure 2) for students to identify the number of inferences they wanted to make per passage and three steps they could take to accomplish that goal. The form also included a graph template for students to chart their progress in each session.

Figure 2.

Goal setting form.

Procedure

The two learning strategies classes at School A and the one class of students at School B were randomly assigned to the three treatment conditions: inference instruction only (IO), inference + individual goal setting (IIG), and inference + group goal setting (IGG). Although random assignment of classes is a less rigorous research design than random assignment of individual students (What Works Clearinghouse, 2014), cluster-randomized designs can be considered necessary (a) to minimize contamination between treatments within the same classroom or (b) when the treatment is intended at the level of the cluster (Hedges & Rhoads, 2009). Both scenarios were present in this study: It was necessary to (a) minimize contamination among students assigned to IO or IIG and (b) implement IGG with one intact class.

Inference only (IO)

In the first of the seven sessions, the interventionists started by building students’ background knowledge of what an inference was. They defined inference (i.e., using stated information to figure out a piece of information that was not directly stated) and then wrote four sentences on the board to demonstrate making inferences and to practice doing so with the students. For the example sentence, “Harold dribbled down the court and passed the ball to Anne,” the interventionists explained that the information about dribbling, being on a court, and passing a ball could be used to infer that Harold and Anne were playing basketball.

Next, the interventionists distributed the graphic organizer and explained how it would be used to record information stated in a passage (e.g., dribbling and passing a ball) in the things you know for sure box and information that could be inferred (e.g., playing basketball) in the things you infer or figure out box. They then distributed a passage and provided a brief introduction to the fictional story and three key words. The interventionists read the passage aloud to students while stopping periodically to model completing the graphic organizer. The interventionists performed a think aloud, verbalizing why they recorded some interesting information that connected to one or more pieces of information stated earlier in the text or that seemed to suggest something else that was not directly stated by the author. The interventionists also explained how they might use a sentence stem from the list to help compose an inference if they were not sure how to express it. Using the stems was encouraged but not required.

In Session 2, the interventionists reviewed with the students what an inference was, how inferences were made from stated information, how both the stated and inferred information were recorded on the graphic organizer, and how to use the list of stems when necessary. They then distributed a new passage and blank graphic organizer before providing an introduction to the topic and time period of the informational narrative as well as the key words. The interventionists and the students alternated reading the passage aloud so that the class could stop together to record information on the graphic organizers and generate inferences.

After introducing the passage (alternating fictional and informational narratives) and about three key words, the interventionists also conducted Session 3 in a whole-group fashion. However, students now were asked to generate their inferences independently whenever the class stopped to record information on their graphic organizers. Although the interventionists always introduced the passage and key words, students gradually assumed greater responsibility for making inferences. In Sessions 4 and 5, students worked in pairs to complete the graphic organizers, and they worked independently in the final Sessions 6 and 7. Students always received a new graphic organizer and had access to the list of inference stems. In addition, they received feedback on their performance each session so that they knew when they were making valid inferences (i.e., inferences connected to information stated in the text).

Inference + individual goal setting (IIG)

Instruction began and proceeded in the same fashion as described for the IO group. The only difference was that, starting in Session 2, students in the IIG group were asked to set a goal for the number of inferences they thought they could make by considering how well they understood and contributed to the process of making inferences from the previous session. Each student recorded that number as his or her individual goal on the goal setting form and then worked with the interventionist to develop three steps for reaching that goal. For example, a student might use the stems, confer with a partner, and reread the passage. After receiving their graded organizers each session, students recorded the number of their correct inferences on the graph. A student who met his or her goal for two consecutive sessions then increased the goal. A student who did not meet his or her goal for two consecutive sessions worked with the interventionist either to revise the steps to take or decrease the goal.

Inference + group goal setting (IGG)

As with the IIG group, instruction in the IGG group began and proceeded in the same fashion as described for the IO group. The only difference was that, in Session 2, the IGG group collectively set a goal for how many inferences they thought they could make. All students had to agree to the same goal. That same number was recorded on everyone’s goal setting form, and the class then worked with the interventionist to develop three steps for how they could reach the goal. After receiving their graded graphic organizers each session, students recorded their own number of correct inferences on the graph. If all students in the class met the group goal for two consecutive sessions, they collectively increased the goal. If everyone did not meet the group goal for two consecutive sessions, they worked with the interventionist either to revise their steps or goal.

Dosage and duration

Across groups, intervention sessions were provided 1 to 2 times per week based on other school or facility scheduling demands. The seven sessions spanned 7 weeks in Building A (groups IO and IIG) and 5 weeks in Building B (group IGG). The review of inference instruction in Sessions 2 through 7 lasted 3 to 5 min, and the text reading with periodic inference making required 30 to 40 min of class time. In the IIG and IGG groups, the review of goals lasted an average 2 min per session.

Fidelity of Implementation

All sessions across groups were audio recorded, and the recordings subsequently were coded by two separate raters, neither of whom was involved in delivering the instruction. Poor quality audio resulted in the loss of one session from each of the IO and IIG groups (14% of the fidelity data from those groups), so a total of 19 sessions (90% of all fidelity data) were coded for fidelity to treatment protocols. Raters disagreed on only one code, resulting in 99% inter-rater reliability. Using the more conservative rating, overall fidelity was 99% with a range of 96% in the IIG group to 100% in the IO and IGG groups.

Results

Descriptive data for all dependent variables are provided in Table 2. Given the small sample, different group sizes, and use of clustered-randomized assignment, the nonparametric Kruskal–Wallis one-way analysis of variance was used to evaluate whether the pre- and posttest data differed significantly among the IO, IIG, and IGG groups (Kruskal & Wallis, 1952). The Kruskal–Wallis H test is similar to a Mann–Whitney U test but allows for the comparison of more than two groups. Results of the pretest analysis indicated there were no statistically significant differences between groups on the total easyCBM score or the inference and evaluation easyCBM items specifically (see Table 3 for mean ranks and Chi-square values).

Table 2.

Descriptive Data on Measures.

Measure	IO (n = 9)	IIG (n = 7)	IGG (n = 8)	Total (N = 24)
Measure	M (SD)	M (SD)	M (SD)	M (SD)
easyCBM pretest total (20 items)	10.67 (3.64)	11.29 (3.90)	12.38 (2.39)	11.43 (3.23)
easyCBM pretest literal (7 items)	4.56 (1.42)	5.57 (0.98)	5.25 (1.28)	5.08 (1.26)
easyCBM pretest inference (7 items)	3.44 (1.59)	3.00 (1.41)	4.13 (1.73)	3.55 (1.55)
easyCBM pretest evaluation (6 items)	2.67 (1.12)	2.71 (1.98)	3.00 (1.07)	2.79 (1.32)
Session 2^a attempts	3.57 (0.79)	3.88 (1.07)	4.57 (0.98)	4.00 (0.98)
Session 2 valid inferences	2.14 (1.35)	2.57 (1.51)	2.57 (2.44)	2.43 (1.71)
Session 3 attempts	4.50 (0.53)	5.00 (2.00)	10.71 (2.06)	6.72 (3.21)
Session 3 valid inferences	2.75 (1.28)	2.50 (1.76)	10.29 (1.50)	5.20 (3.86)
Session 4 attempts	4.50 (1.07)	3.50 (0.84)	10.13 (5.67)	6.27 (4.40)
Session 4 valid inferences	3.50 (0.76)	2.50 (0.55)	7.50 (3.89)	4.68 (3.12)
Session 5 attempts	4.56 (0.73)	4.50 (1.05)	11.28 (4.72)	6.91 (4.22)
Session 5 valid inferences	3.44 (1.51)	3.17 (1.33)	9.88 (2.80)	5.61 (3.64)
Session 6 attempts	4.71 (0.76)	4.43 (0.79)	7.38 (2.56)	5.59 (2.06)
Session 6 valid inferences	4.00 (1.00)	2.86 (0.38)	6.75 (2.66)	4.64 (2.31)
Session 7 attempts	4.88 (0.83)	3.86 (1.57)	8.88 (3.48)	5.97(3.04)
Session 7 valid inferences	4.25 (1.04)	3.29 (1.80)	8.00 (4.04)	5.27 (3.21)
easyCBM posttest total (20 items)	13.56 (3.28)	13.71 (4.15)	15.50 (2.33)	14.26 (3.21)
easyCBM posttest literal (7 items)	5.78 (1.48)	6.29 (0.95)	5.50 (1.20)	5.84 (1.21)
easyCBM posttest inference (7 items)	4.33 (1.41)	4.14 (2.04)	4.13 (1.26)	4.21 (1.44)
easyCBM posttest evaluation (6 items)	3.44 (1.59)	3.29 (1.50)	5.88 (0.35)	4.21 (1.68)

Note. IO = inference instruction only; IIG = inference + individual goal setting; IGG = inference + group goal setting.

Session 1 is not included because it was conducted as teacher modeling.

Table 3.

Chi-Square Tests.

Dependent measure	Treatment group			χ²
	IO	IIG	IGG
	Mean rank	Mean rank	Mean rank
easyCBM pretest total	10.94	12.64	14.13	0.87
easyCBM pretest literal	9.78	15.00	13.38	2.48
easyCBM pretest inference	12.28	10.14	14.81	1.70
easyCBM pretest evaluation	11.89	11.71	13.88	0.49
Session 2^a attempts	8.21	10.50	14.29	3.72
Session 2 valid inferences	10.43	11.50	11.07	0.11
Session 3 attempts	7.25	7.83	18.00	13.71***
Session 3 valid inferences	7.94	6.92	18.00	13.68***
Session 4 attempts	9.81	5.58	17.63	13.34***
Session 4 valid inferences	9.94	4.75	18.13	16.16***
Session 5 attempts	8.17	7.83	19.44	15.53***
Session 5 valid inferences	8.56	7.50	19.25	14.40***
Session 6 attempts	8.79	6.93	17.88	14.23***
Session 6 valid inferences	10.71	5.29	17.63	14.73***
Session 7 attempts	17.88	7.07	10.44	11.06**
Session 7 valid inferences	10.81	7.79	16.88	7.26*
easyCBM posttest total	10.67	11.64	15.31	1.99
easyCBM posttest literal	12.72	14.86	10.19	1.80
easyCBM posttest inference	13.11	12.29	12.00	0.119
easyCBM posttest evaluation	9.22	8.50	19.69	13.18***

Note. IO = inference instruction only; IIG = inference + individual goal setting; IGG = inference + group goal setting.

Session 1 is not included because it was conducted as teacher modeling.

p ≤ .05. **p ≤ .01. ***p ≤ .001.

Productive Tasks

There were no statistically significant differences among the treatment groups on the number of attempted or valid inferences in Session 2. This was the first time students were involved in making inferences, but the interventionists led this as a whole-group lesson and had not yet introduced goal setting to the IIG or IGG groups. When students were more self-directed in making inferences during Sessions 3 through 7, the IGG group significantly outperformed both the IO and IIG groups on the number of inferences attempted, χ²(2) ranged from 11.06, p = .004, to 15.53, p < .001 and the number of those attempts that were accurately linked to information stated directly in the text; χ²(2) ranged from 7.26, p = .03, to 16.16, p < .001. A Benjamini–Hochberg correction (Benjamini & Hochberg, 1995) was applied to control for the false discovery rate in multiple hypothesis testing. The resulting q value was .023, confirming the results were significant.

Students in the IO group made mostly steady improvements in the number of valid inferences they could make on a passage, regardless of whether it was a fictional or informational narrative or whether they were working with peers or individually. They progressed from an average 2.14 inferences in Session 2 to 4.25 inferences in Session 7, with only one small decline from Session 4 (average 3.50 inferences) to Session 5 (average 3.44 inferences). The IIG group had more variable performance with two periods of decline (from Sessions 2–3 and 5–6), which coincided with the transitions from teacher-directed to partner work and from partner work to completely independent work. The IGG group demonstrated a large increase in the number of valid inferences made once goal setting began (average 2.57 inferences in Session 2 to 10.29 inferences in Session 3). The latter was the highest number of valid inferences made in any session. Thereafter, performance fluctuated with the lowest average of 6.75 inferences in Session 6 and a final average of 8.00 inferences in Session 7.

Receptive Tasks

The significantly better performance of the IGG group was maintained in the posttest only on the evaluation (or high-level inference) items of the easyCBM, χ²(2) = 13.18, p = .001. Again, the Benjamini–Hochberg correction confirmed the significant finding, q = .023. The overall easyCBM score as well as the scores on the literal and inference items specifically were not significantly different across groups. However, the standard deviations suggest there was less variability in the performance of the IGG students.

Despite the lack of significant differences in their overall posttest scores, it was apparent that all groups improved. Paired samples t tests confirmed the within-group gains on the overall comprehension score were significant, IO: t(8) = 2.76, p = .025; IIG: t(6) = 3.97, p = .007; IGG: t(7) = 4.35, p = .003. The improvement was specific to the inference items for the IO, t(8) = 2.87, p = .021, and IIG groups, t(6) = 2.83, p = .030. It was specific to the evaluation items for the IGG group, t(7) = 7.22, p < .001.

Goal Attainment

Data on the number of students in the IIG and IGG groups who met their goal for each of the Sessions 3 through 7 (the lessons which incorporated goal setting and required students to be more self-directed in making inferences) were analyzed descriptively. Means are presented in Table 4. The IGG group had a higher proportion of students meeting the group goal in each session (75%–100%) than the proportion of students in the IIG group meeting their individual goals (0%–57%).

Table 4.

Average Number of Students Who Met Their Goal Each Session.

Intervention session	IO (n = 9)	IIG (n = 7)	IGG (n = 8)
Session^a 2	N/A	N/A	N/A
Session 3	N/A	3	7
Session 4	N/A	3	7
Session 5	N/A	4	8
Session 6	N/A	0	6
Session 7	N/A	2	8

Note. IO = inference instruction only; IIG = inference + individual goal setting; IGG = inference + group goal setting; N/A = not applicable.

Session 1 is not included because it was conducted as teacher modeling, and Session 2 did not incorporate goal setting.

Discussion

Because reading achievement in the middle grades and beyond hinges on students’ ability to glean more than a surface-level understanding of the text (Cain et al., 2001; van den Broek, 1997), the study reported here implemented an inference-making intervention that incorporated a number of important features. To facilitate writing a greater number and variety of inferences (Gygax et al., 2004; van den Broek, 1997), students read longer fictional and informational narrative passages from a typical sixth-grade anthology. Interventionists built or activated background knowledge by introducing the passage and key words before students began reading (Cain et al., 2001). In addition, students used a graphic organizer as an external memory store to overcome reported difficulties with integrating information from across the passage (Barth et al., 2015; Hua & Keenan, 2014), and students had access to inference stems as a support for making inferences without prompting.

Improvements in Productive and Receptive Inference-Making Tasks

In answer to the first research question, findings revealed that middle-grade students with LD who received explicit instruction significantly improved their pre- to posttest performance on a multiple-choice test of reading comprehension. This extends the positive findings of previous studies (Elbro & Buch-Iverson, 2013; Fritschmann et al., 2007; Holmes, 1985; McMaster et al., 2012), which used questions to prompt particular inferences with shorter passages of a single type (either narrative or informational). The gains demonstrated by participating students were attributable to their significant improvement on non-literal items: the inference items for the IO and IIG groups or the evaluation (high-level inference) items for the IGG group. Students’ improvement also might be described as practically significant because their overall performance would have been considered a failing grade (M = 57% correct responses) at pretest but a passing grade (M = 71% correct) at posttest just 5 to 7 weeks later.

The test passages were different at each testing point, and neither passage was used in the intervention sessions. Therefore, students’ performance is interpreted as an indicator of their ability to generalize the knowledge of inference making to a new context for which they did not receive any of the instructional components described above (i.e., building background knowledge, using a graphic organizer, or having the freedom to make unprompted inferences). The multiple-choice items were a receptive task as opposed to the productive act of making inferences without a specific prompt. Across the seven sessions, performance on the production of inferences was variable with the pattern of performance seemingly related to treatment group, which partly answers the second research question.

Effects of Embedded Goal Setting

Students in the IO group, who were not asked to set goals, demonstrated steady improvement in the number of inferences they attempted and the valid number of inferences made. This seemed immune to passage type (fictional or informational narrative) and stage of explicit instruction (teacher-led, partner work, or independent). Embedding individual goal setting did not make a significant difference in the number of inferences attempted or made by the IIG group, but it did increase the variability of their attempts and valid inferences. Declines in the valid number of inferences made coincided with instructional stage changes, suggesting students initially had some difficulty achieving their goals whenever the amount of support decreased. This was not the case for the students in the IGG group. Once they set their first group goal, the IGG students began attempting to make significantly more inferences than the IO or IIG groups and had significantly more valid inferences than the other groups. Although the IGG group’s performance was variable, it was not related to instructional stage change.

Anecdotally, interventionists noticed that students in the IIG group tended to set goals at the extreme high and low values. That is, they either wanted to establish a low bar to ensure they could meet it, or they wanted to set an unrealistically high bar as though in competition with their peers. This difficulty with goal setting among students with LD has been noted by others (Graham & Harris, 1989; Swain, 2005). In the present study, setting the goal as a group seemed to balance out the over- and under-ambitiousness of students because they had to negotiate what everyone was willing and able to do. Those who had that sense of competitiveness applied it to attempting inferences above the number of the group goal. This was an unintended but beneficial strategy for those students. Making more inferences increased the likelihood they would have some correct and would meet or exceed the goal. Those who only tried to make the minimum number of inferences established by the group goal would fail to meet it if any of the inferences were deemed invalid (i.e., not connected to information from the text). Because the entire group had to meet the goal to advance to a new goal, students felt some accountability for altering the steps they would take (see goal setting form in Figure 2) by attempting to write more inferences. This is reflected in the higher proportion of IGG students meeting the goal each session as compared with the IIG group.

The increased practice the students in the IGG group imposed on themselves could have contributed to their significantly better performance on the evaluation items posttest. However, the three treatment groups did not significantly differ in their overall posttest or their inference item performance.

Limitations

This study was conducted in unique settings (self-contained learning strategies classes and a juvenile justice facility), which limited the sample size in each treatment group. In addition, the requirements for the goal setting necessitated using a cluster-randomized design (Hedges & Rhoads, 2009). These circumstances present the possibility that the findings are due to particular characteristics present in this sample and, potentially, not true of or generalizable to the larger population of middle-grade students with reading difficulties. Furthermore, the authors acknowledge there were fewer students with identified disabilities in the IGG group in School B (the juvenile justice facility) as compared with the IO and IIG groups in School A (the public middle school). Juvenile offenders often are under-identified with disabilities (Snyder & Sickmund, 2006; Watson et al., 2009), but it is possible any differences in performance were due to the higher rate of LD within the IO and IIG groups rather than the effects of the goal setting conditions. However, there were no significant differences found at pretest or on the first teacher-guided, whole-group lesson (Session 2). The IGG group’s significantly higher number of attempts and valid inferences were not realized until goal setting initiated in Session 3.

Given that the groups were taught by different interventionists, it is not possible to rule out teacher effects. Fidelity in all conditions was consistently high (96%–100%), thus increasing confidence that instruction was equivalent across groups and sessions. Moreover, an independent rater, who was bind to condition, verified the scoring of all dependent measures with consistently high inter-rater reliability (97%–100%).

Some variability in student performance across sessions or from pre- to posttest could be attributable to passage effects. Although future studies might attempt to use equated passages, the preference in the present study was to evaluate student performance with ecologically valid materials and measures (Bronfenbrenner, 1993). Passages were drawn from a typical school anthology, and the test administered was not unlike the kind a typical classroom teacher might give. With respect to the latter, the pre- and posttest included only one passage each and had few items per type (literal = 7, inference = 7, evaluation = 6). The reported Cronbach’s α range of .63 to .67 (Sáez et al., 2010) for all 20 items is slightly below the liberal minimum of .70 (Nunnally, 1978) but likely would be even lower for the smaller number of disaggregated item types. Hence, the significant finding for the evaluation items needs to be interpreted with caution.

Instructional Implications and Directions for Future Research

The participating middle-grade students with reading difficulties learned to make inferences from text-based information with relatively little instruction. Instruction progressed from modeling, to guided practice, to independent practice in seven sessions lasting 45 to 50 min each. This is similar to the time required for two previously studied interventions (Elbro & Buch-Iverson, 2013; Fritschmann et al., 2007) and less than that for the McMaster et al. (2012) intervention. In contrast to the extant research, students in this study read longer texts and were not reliant on teacher-prepared questions to narrow their inferences. Rather, they were able to write their own inferences, using only a generic graphic organizer and a list of suggested sentence stems. Hence, it is believed the inference intervention implemented could be a feasible practice for classroom teachers. Future research should explore its effectiveness in general education classes. Ideally, these studies would be planned for longer durations and include comparison groups not receiving the intervention as well as standardized measures of student performance and growth.

For two of the treatment groups, goal setting was embedded within the inference intervention. Classroom teachers have expressed concern that they lack the time or knowledge to address self-regulation skills (Stang et al., 2009; Wehmeyer et al., 2000), but the simple approach reported here took a minimal amount of time each session—about 2 min. Students in the IIG group did not set very appropriate goals when asked to do so individually, and the goal setting did not contribute to significantly better performance. In fact, the IO group without any goal setting was making more steady progress that might, over an extended period of time, have demonstrated an advantage over the IIG group. If that hypothesis is borne out in future research, it would render the 2 min per day devoted to individual goal setting an unnecessary loss of instructional time that could add up to several hours over the course of a semester or year.

However, the group goal setting seemed to be more beneficial for encouraging practice and self-reflection on students’ efforts and progress, which has been shown to improve self-efficacy in reading and writing (Schunk, 2003). This warrants further study for its potential relationship to distal student outcomes such as overall achievement and educational attainment. As Schunk (2003) stated, “People do not attempt to attain what they believe is impossible” (p. 164). For middle-grade students who have experienced repeated and increasing difficulty with the inference-making expectations of course work and accountability tests, it may be motivating to learn an intervention that is transferable to different texts and with which they are able to make noticeable progress.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Alonzo

Tindal

Ulmer

Glasgow

(2006). easyCBM online progress monitoring assessment system. Eugene, OR: Center for Educational Assessment Accountability. Available from http://easycbm.com

Baker

(2002). Metacognition in comprehension instruction. In Block

C. C.

Pressley

(Eds.), Comprehension instruction: Research-based best practices (pp. 77–95). New York, NY: Guilford Press.

Baker

S. K.

Chard

D. J.

Ketterlin-Geller

L. R.

Apichatabutra

Doabler

(2009). Teaching writing to at-risk students: The quality of evidence for self-regulated strategy development. Exceptional Children, 75, 303–318. doi:10.1177/001440290907500303

Barth

A. E.

Barnes

Francis

Vaughn

York

(2015). Inferential processing among adequate and struggling adolescent comprehenders and relations to reading comprehension. Reading and Writing, 28, 587–609. doi:10.1007/s11145-014-9540-1

Benjamini

Hochberg

(1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B (Methodological), 57, 289–300.

Bronfenbrenner

(1993). Ecological models of human development. In Gauvain

Cole

(Eds.), Reading on the development of children (2nd ed., pp. 37–43). New York, NY: Freeman.

Cain

Oakhill

Barnes

M. A.

Bryant

P. E.

(2001). Comprehension skill, inference-making ability, and their relation to knowledge. Memory & Cognition, 29, 850–859. doi:10.3758/BF03196414

Drerup

Croysdale

Hoffman

(2008). Patterns of behavioral health conditions among adolescents in a juvenile justice system. Professional Psychology: Research and Practice, 39, 122–128. doi:10.1037/07357028.39.2.122

Elbro

Buch-Iverson

(2013). Activation of background knowledge for inference making: Effects on reading comprehension. Scientific Studies of Reading, 17, 435–452. doi:10.1080/10888438.2013.774005

10.

Fritschmann

N. S.

Deshler

D. D.

Schumaker

J. B.

(2007). The effects of instruction in an inference strategy on the reading comprehension skills of adolescents with disabilities. Learning Disability Quarterly, 30, 245–262. doi:10.2307/25474637

11.

Gillespie

Graham

(2014). A meta-analysis of writing interventions for students with learning disabilities. Exceptional Children, 80, 454–473. doi:10.1177/0014402914527238

12.

Graesser

A. C.

Singer

Trabasso

(1994). Constructing inferences during narrative text comprehension. Psychological Review, 101, 371–395.

13.

Graham

Harris

K. R.

(1989). Cognitive training: Implications for written language. In Hughes

Hall

(Eds.), Cognitive behavioral psychology in the schools: A comprehensive handbook (pp. 247–279). New York, NY: Guilford Press.

14.

Gygax

Garnham

Oakhill

(2004). Inferring characters’ emotional states: Can readers infer specific emotions? Language and Cognitive Processes, 19, 613–639. doi:10.1080/01690960444000016

15.

Harris

K. R.

Friedlander

B. D.

Saddler

Frizzelle

Graham

(2005). Self-monitoring of attention versus self-monitoring of academic performance: Effects among students with ADHD in the general education classroom. The Journal of Special Education, 39, 145–156. doi:10.1177/00224669050390030201

16.

Hedges

Rhoads

(2009). Statistical power analysis in education research (NCSER 2010-3006). Washington, DC: National Center for Special Education Research, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncser/

17.

Holmes

B. C.

(1985). The effects of a strategy and sequenced materials on the inferential comprehension of disabled readers. Journal of Learning Disabilities, 18, 542–546. doi:10.1177/002221948501800909

18.

Hua

A. N.

Keenan

J. M.

(2014). The role of text memory in inferencing and in comprehension deficits. Scientific Studies of Reading, 18, 415–431. doi:10.1080/10888438.2014.926906

19.

Johnston

A. M.

Barnes

M. A.

Desrochers

(2008). Reading comprehension: Developmental processes, individual differences, and interventions. Canadian Psychology, 49, 125–132. doi:10.1037/0708-5591.49.2.125

20.

Kendeou

van den Broek

Helder

Karlsson

(2014). A cognitive view of reading comprehension: Implications for reading difficulties. Learning Disabilities Research & Practice, 29, 10–16. doi:10.1111/ldrp.12025

21.

Kispal

(2008). Effective teaching of inference skills for reading: Literature review (Research Report DCSF-RR031). Berkshire, UK: Department for Children, Schools, and Families; National Foundation for Educational Research. Retrieved from www.dcsf.gov.uk/research

22.

Konrad

Fowler

C. H.

Walker

A. R.

Test

D. W.

Wood

W. M.

(2007). Effects of self-determination interventions on the academic skills of students with learning disabilities. Learning Disability Quarterly, 30, 89–113. doi:10.2307/30035545

23.

Kruskal

W. H.

Wallis

W. A.

(1952). Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47, 583–621. doi:10.1080/01621459.1952.10483441

24.

McMackin

M. C.

Witherell

N. L.

(2005). Different routes to the same destination: Drawing conclusions with tiered graphic organizers. Reading Teacher, 59, 242–252. doi:10.1598/RT.59.3.4

25.

McMaster

K. L.

van den Broek

Espin

White

M. J.

Rapp

D. N.

Kendeou

. . . Carlson

(2012). Making the right connections: Differential effects of reading intervention for subgroups of comprehenders. Learning and Individual Differences, 22, 100–111. doi:10.1016/j.lindif.2011.11.017

26.

National Assessment Governing Board & U.S. Department of Education. (2012). Reading framework for the 2013 National Assessment of Educational Progress. Washington, DC: Author. Retrieved from https://www.edpubs.gov/document/ed005373p.pdf

27.

Nunnally

J. C.

(1978). Psychometric theory (2nd ed.). New York, NY: McGraw-Hill.

28.

Oakhill

Cain

(2012). The precursors of reading ability in young readers: Evidence from a four-year longitudinal study. Scientific Studies of Reading, 16, 91–121. doi:10.1080/10888438.2010.529219

29.

Raskind

M. H.

Goldberg

R. J.

Higgins

E. L.

Herman

K. L.

(1999). Patterns of change and predictors of success in individuals with learning disabilities: Results from a twenty-year longitudinal study. Learning Disabilities Research & Practice, 14, 35–49. doi:10.1207/sldrp1401_4

30.

Sáez

Park

Nese

J. F. T.

Jamgochian

Lai

C. F.

Anderson

. . . Tindal

(2010). Technical adequacy of the easyCBM reading measures (Grades 3-7), 2009-2010 version (Technical Report No. 1005). Eugene, OR: Behavioral Research and Teaching. Available from http://easycbm.com

31.

Schunk

D. H.

(2003). Self-efficacy for reading and writing: Influence of modeling, goal setting, and self-evaluation. Reading & Writing Quarterly: Overcoming Learning Difficulties, 19, 159–172. doi:10.1080/10573560308219

32.

Shogren

K. A.

Palmer

S. B.

Wehmeyer

M. L.

Williams-Diehm

Little

T. D.

(2012). Effect of intervention with the self-determined learning model of instruction on access and goal attainment. Remedial and Special Education, 33, 320–330. doi:10.1177/0741932511410072

33.

Snyder

H. N.

Sickmund

(2006). Juvenile offenders and victims: 2006 national report. Washington, DC: Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention, U.S. Department of Justice. Available from http://www.ojjdp.gov

34.

Stang

K. K.

Carter

E. W.

Lane

K. L.

Pierson

M. R.

(2009). Perspectives of general and special educators on fostering self-determination in elementary and middle schools. The Journal of Special Education, 43, 94–106. doi:10.1177/0022466907313452

35.

Swain

K. D.

(2005). CBM with goal setting: Impacting students’ understanding of reading goals. Journal of Instructional Psychology, 32, 259–265.

36.

Test

D. W.

Fowler

C. H.

Richter

S. M.

White

Mazzotti

Walker

A. R.

. . . Kortering

(2009). Evidence-based practices in secondary transition. Career Development for Exceptional Individuals, 32, 115–128. doi:10.1177/0885728809336859

37.

Thayer

J. F.

Hansen

A. L.

Saus-Rose

Johnsen

B. H.

(2009). Heart rate variability, prefrontal neural function, and cognitive performance: The neurovisceral integration perspective on self-regulation, adaptation, and health. Annals of Behavioral Medicine, 37, 141–153. doi:10.1007/s12160-009-9101-z

38.

Thompson

L. L.

Whitmore

E. A.

Raymond

K. M.

Crowley

T. J.

(2006). Measuring impulsivity in adolescents with serious substance and conduct problems. Assessment, 13, 3–15. doi:10.1177/1073191105282247

39.

van den Broek

(1997). Discovering the cement of the universe: The development of event comprehension from childhood to adulthood. In van den Broek

Bauer

P. W.

Bourg

(Eds.), Developmental spans in event comprehension and representation (pp. 321–342). Mahwah, NJ: Lawrence Erlbaum.

40.

Watson

A. C.

Kelly

B. L.

Vidalon

T. M.

(2009). Examining the meaning attached to mental illness and mental health services among justice system-involved youth and their parents. Qualitative Health Research, 19, 1087–1099. doi:10.1177/1049732309341202

41.

Wehmeyer

M. L.

Agran

Hughes

(2000). A national survey of teachers’ promotion of self-determination and student directed learning. The Journal of Special Education, 34, 58–68. doi:10.1177/002246690003400201

42.

Wexler

Reed

D. K.

Sturges

(2015). Reading practices in the juvenile correctional facility setting: Incarcerated adolescents speak out. Exceptionality, 23, 100–123. doi:10.1080/09362835.2014.986602

43.

What Works Clearinghouse. (2014). Procedures and standards handbook (Version 3.0). Washington, DC: Institute of Education Sciences, U.S. Department of Education.