Efficacy of Self-Regulated Strategy Development Instruction for Developing Writers With and Without Disabilities in Rural Schools: A Randomized Controlled Trial

Abstract

A workshop with virtual consultation practice-based professional development model for self-regulated strategy development persuasive writing instruction was evaluated in a randomized controlled trial. Nineteen general education teachers and 564 Grade 5 and 6 students in 16 low-wealth rural schools participated. Following training, teachers provided instruction in their inclusive classrooms. Results indicated that students receiving instruction improved in the number of persuasive elements and words written when compared with students in the control group. Differences in element type were evaluated with largest effects noted for reasons and explanations. Student group differences were noted with nonstruggling students having greater gains than struggling students.

Keywords

writing instruction professional development rural schools

Forty-three percent of school districts in the United States are located in rural areas, serving approximately 20% of all public school students (Johnson, Showalter, Klein, & Lester, 2014). Like many schools in the United States, rural schools are challenged with teacher recruitment and retention difficulties, licensure issues, increasing student population diversity, rapidly changing economic structures, and poverty (Mattingly, Carson, & Schaefer, 2014). Furthermore, students in rural schools are noted to have lower academic success than peers from metropolitan areas (Rojewski, 1999). On the 2012 National Assessment of Educational Progress (NAEP) test, only 32% and 33% of rural students performed at or above the Proficient level in reading in Grades 4 and 8, respectively (National Center for Education Statistics, 2013). Results for the 2011 NAEP writing assessment were weaker; only 24% of Grade 8 and 12 rural students performed at or above the Proficient level (National Center for Education Statistics, 2012). To improve rural student outcomes, providing professional development for teachers is an ongoing research priority (Brendle, 2015) and is the focus of our intervention study.

Given poor student outcomes in writing, developing effective professional development for writing instruction is timely. However, current methods of in-person professional development are generally not feasible for rural teachers. Interfering factors include a lack of professional development resources, geographic isolation, and unavailability of substitute teachers for release time. In light of these challenges, Seltzer and Himley (1995) recommended that professional development session organizers in rural school systems (a) invite school teams (e.g., teachers, specialists, and principals) to participate, (b) promote ownership of the professional development agenda, (c) plan for ongoing technical assistance, (d) make gradual changes, and (e) focus on teaching and learning strategies that foster meaningful connections. The current study, therefore, was situated in a model of change dependent on these factors and on one well-established research-based intervention and professional development model, practice-based professional development for self-regulated strategy development (Harris, Graham, & Mason, 2003).

Writing Instruction

Standards initiatives, such as the U.S. Common Core State Standards (Common Core State Standards Initiative [CCSSI], 2012), for written expression require a high level of writing skill. Graham (2006) noted that strategy instruction has been documented as effective in improving the number of elements written, with an effect size (ES) of 1.15 for group comparison studies and a percentage of nonoverlapping data (PND) of 95% for single-case studies. Additional seminal literature reviews and meta-analyses have documented the effectiveness of explicit strategy instruction for improving writing (e.g., Graham, McKeown, Kiuhara, & Harris, 2012; Graham & Perin, 2007).

Self-Regulated Strategy Development (SRSD)

SRSD instruction promotes independent use of task-specific writing strategies by teaching students cognitive and self-regulation strategies in tandem so they can better understand and regulate the writing process. SRSD instruction is situated within an iterative writing process model, as described by Flower and Hayes (1980), and within a sociocognitive model of learning (e.g., Vygotsky, 1986). In SRSD, students are active participants in the learning process and student effort is acknowledged and rewarded (Harris et al., 2012). Six scaffolded instructional stages facilitate the student’s mastery of strategy use: develop background knowledge, discuss it, model it, memorize it, use guided practice, and perform independent practice (Harris, Graham, Mason, & Friedlander, 2008). Students’ self-regulation of strategy use is supported by teaching students to set goals, self-monitor, self-instruct, and self-reinforce (Harris et al., 2012). Graham and Perin (2007) noted that SRSD significantly improves the structure of the writing of students in Grades 4 through 12 when compared with other types of strategy instruction and control conditions. Similar findings were noted in Mason and Graham’s (2008) review of writing instruction for Grade 4 through 12 students with learning disabilities; SRSD instruction resulted in large gains in writing quality when compared with that of students in a business-as-usual control group (ES = 1.26–1.35) and in a separate review documenting improved writing skills for students with emotional and behavioral disorders (McKeown, Fitzpatrick, & Sandmel, 2014).

Two SRSD studies are particularly relevant for the instructional delivery as selected for the current study. Each study included four to five lessons focused on teaching students to write a short paper response, a “quick write.” In a quasi-experimental group study, Mason, Kubina, Kostewicz, Mong Cramer, and Datchuk (2013) investigated the effects of SRSD supplemental instruction, delivered by a graduate assistant, on the persuasive writing of 23 low-achieving urban Grade 7 and 8 students with and without disabilities. Students in treatment increased persuasive writing performance as indicated by large effect sizes (ES = 1.38 for number of parts and ES = 1.49 for organization quality) when compared with a comparison group of average-achieving peers. In the second study, SRSD for planning and writing an informative paper was evaluated in a single-case multiple-baseline study across four Grade 6 inclusive science classrooms in a suburban school (Benedek-Wood, Mason, Wood, Hoffman, & MacGuire, 2014). Results indicated the 78 students’ performance in writing quality and number of words written improved with large PND effects.

Practice-Based Professional Development (PBPD) for SRSD

PBPD for SRSD writing instruction specifies eight critical components for professional development in writing instruction: (a) the value of written expression, (b) procedures for developing written expression, (c) evidence-based principles for instruction and assessment of written expression, (d) procedures for differentiation, (e) opportunities for scaffolded practice in teaching, (f) procedures for sustainability, (g) procedures for in-school and district leadership, and (h) presentation of the feasibility of the model (Harris et al., 2012). Two foundational studies informed our PBPD design (Harris, Graham, & Adkins, 2015; Harris et al., 2012). In a randomized controlled trial study, Harris et al. (2015) examined the effects of PBPD for SRSD small-group instruction with Grade 2 teachers in suburban schools. PBPD was delivered for 12 to 14 hr spread across 2 days in the teachers’ schools. Results indicated large gains in story quality and number of elements at posttest and maintenance (ES = 0.89–1.65). In the second randomized controlled trial (Harris et al., 2012), results were replicated for whole-class instruction with Grade 2 and 3 teachers in three schools. After SRSD, students increased the number of elements in their stories (ES = 1.82) and opinion essays (ES = 2.02).

Current Study

The current study for Grade 5 and 6 students differed from prior PBPD for SRSD research in four significant ways:

PBPD was provided across multiple states and geographic areas in low-wealth schools.

PBPD time was reduced to 1 day and was not conducted in teachers’ schools.

Ongoing support was provided via virtual consultation.

Teachers’ instructional delivery was reduced to five 45-min lessons.

Premises for effective professional development, as described by Ball and Cohen (1999), Desimone (2011), and Borko (2004), were adopted. For example, to address barriers in rural schools, we invited school teams to a central location in their state to participate in PBPD during the summer months. A team-based approach with teachers and administrators from the same school collectively participating was implemented (Borko, 2004). In-person PBPD and the teachers’ instructional manual used in the current study embedded the eight components from the Harris et al. (2012) model (see PBPD in the “Method” section).

Given a reduced amount of time for in-person professional development and the inability to support teachers in their classrooms, we extended training and support by including virtual consultation. This was critical, as teachers who participate in professional development activities with no follow-up support are unlikely to implement a new practice successfully (Cornett & Knight, 2009). Virtual learning has been documented to be as effective as in-person professional development for improving teacher learning (Fisher, Schumaker, Culbertson, & Deshler, 2010; Mason et al., 2013).

We selected one genre, persuasive writing, for the focus of the current study. Persuasive writing is a particularly challenging form of written expression and one that is not well developed until late elementary school (Nippold, Ward-Lonergan, & Fanning, 2005). Two writing strategies were selected as the focus for SRSD instruction (Harris et al., 2008). The first strategy, POW, is a general 3-step planning strategy: (a) Pick an idea or side of a topic, (b) Organize ideas by planning with notes, and (c) Write and say more by modifying and improving the original plan. The second strategy, TREE, helps students formulate basic elements of persuasion: (a) write a convincing Topic sentence, (b) write at least three Reasons why you believe the topic sentence, (c) write Explanations to support each reason, and (d) wrap it up with a good Ending sentence. The effectiveness of SRSD for POW + TREE in persuasive writing was first established in randomized controlled trial experimental studies in elementary settings with struggling writers (e.g., Graham, Harris, & Mason, 2005; Harris, Graham, & Mason, 2006). Subsequently, SRSD for POW + TREE was validated in quasi-experimental (e.g., Mason et al., 2013) and multiple single-case design studies with Grade 6 through 8 students with and without learning disabilities, attention-deficit/hyperactivity disorder, and emotional and behavioral disorders (see Mason & Kubina, 2011).

In the current randomized controlled trial study, we examined the effects of our PBPD model for SRSD instruction on students’ writing, as measured by elements written and on students’ writing skill-level differences. The following three research questions were proposed:

Research Question 1: Did PBPD for SRSD persuasive writing instruction relate to elements of students’ persuasive written expression as measured by the total number of reasons written, total number of explanations written, or the inclusion of a topic or an ending sentence?

Research Question 2: What is the effect of teacher assignment to the PBPD for SRSD intervention on students’ total number of words written and total number of persuasive elements written?

Research Question 3: Does the intervention effect on total number of persuasive elements written vary due to students’ writing ability?

Method

The feasibility of PBPD in the rural context was evaluated in the context of a larger development study, the Rural Early Adolescent Learning Program (Project REAL; see Farmer, Hall, Petrin, Hamm, & Dadisman, 2010; Hamm, 2017; Hamm et al., 2010), focused on developing teachers’ knowledge and skills for creating school environments that support positive adjustment for students in early adolescent transition. Project REAL was conducted across seven states and 35 schools over a 2-year period. In the first year of the study, PBPD training and virtual consultation for SRSD persuasive writing instruction were provided to participating schools’ general education teachers, classroom support specialists (e.g., special educators, speech–language pathologists), and school administrators in a 2-day summer institute. The first day of the institute focused on writing instruction; the second day focused on positive adjustment. After the 1-day training for SRSD, in the first weeks of the school year, but prior to the implementation of the positive adjustment intervention, teachers delivered writing assessments followed by SRSD for POW + TREE instruction in their general education classrooms. Virtual consultation support was provided during instructional periods. Pre- and postinstruction data collection on writing and all classroom SRSD instruction was concluded prior to implementation of procedures and data collection for the larger study.

Participants and Setting

Participants were 19 inclusive general education teachers and 564 Grade 5 and 6 students in 16 schools across four U.S. states located in the Midwest, Southeast, and Southwest. All 16 schools were eligible for U.S. Department of Education’s (2016) Rural and Low-Income School Program (RLISP) by having a locale code of 6, 7, or 8 and at least 20% of students from families living below the federal poverty level. Ethnicity included (a) 100% Caucasian students in eight Midwestern schools, (b) 100% African American students in four Southeastern schools, (c) 100% Hispanic students in two Southwestern schools, and (d) 60% Caucasian/40% Hispanic students in two Southwestern schools.

Five hundred and ninety-two students provided consent, for a total of 58.65% female and 41.35% male students in Grades 5 (95.49%) and 6 (4.51%; see Table 1). Students with disabilities receiving special education services were not excluded from the study; however, the decision was made to not identify or aggregate data into a single group of students identified as receiving special education services because (a) diagnosis categories vary from state to state (Hallahan et al., 2007) and (b) the focus of the study was on outcomes for persuasive writing, specifically for students identified as struggling and nonstruggling writers in the context of inclusive classroom instruction and assessment (see the “Assignment Procedures” section). The majority (80%) of the intervention teachers had more than 10 years of experience teaching; 40% with a master’s degree. Control group teachers’ averages were comparable (50% with greater than 10 years’ teaching experience and 44% with a master’s degree). All teachers were certified in their area/grade level.

Table 1.

Student Demographic and Descriptive Data.

Demographic/descriptive data	Control group (n = 266)				Intervention group (n = 326)
Demographic/descriptive data	n	%	M	SD	n	%	M	SD
Gender
Female	156	58.65			171	52.45
Male	110	41.35			155	47.55
Grade
Fifth	254	95.49			310	95.09
Sixth	12	4.51			16	4.91
Ability
Struggling	48	18.05			47	14.42
Not struggling	218	81.95			279	85.58
Pretest total
Words	266		87.26	40.22	326		92.14	48.24
Elements	266		4.25	2.14	326		5.09***	2.35
Reasons	266		2.26	1.36	326		2.73***	1.50
Explanations	266		0.77	1.02	326		1.20***	1.26
Pretest % yes
Topic	266		84.59	36.18	326		78.22*	41.34
Ending	266		37.22	48.43	326		37.42	48.47

p < .05. ***p < .001.

Assignment Procedures

Eight matched-paired schools were randomly assigned to one of two groups: SRSD instruction or business-as-usual control. Schools were matched on a series of demographic features, such as geographic location by state, size of student population, student achievement as determined by state testing data, and student poverty (i.e., free/reduced-price lunch status). The intervention was implemented at the school level; all teachers in each intervention school took part in PBPD. Current writing practice for control schools was not accounted for; therefore, control group instruction was considered a business-as-usual condition. Intervention materials were made available to the control schools at the end of the research project.

We calculated students’ writing ability by examining the mean and standard deviation of total elements written (see the “Measures” section) at pretest. Standard deviation cut-points have been used in prior SRSD research to identify struggling and/or at-risk writers. For example, (a) two thirds SD below the mean on a standardized story writing assessment was used in Graham et al. (2005) and in Harris et al. (2006), and (b) 1.5 SD below the mean on a standardized fluency measure was used in Mason et al. (2013). In these studies, assessment for establishing student need for instruction based on ability was directly related to type of instruction (i.e., story writing or quick writing). In the current study, we used students’ persuasive writing at pretest to determine ability to write in this genre. Students scoring more than 1 SD below the mean were considered struggling (n = 95), whereas their peers were considered nonstruggling writers (n = 497).

Intervention

In the first state to receive PBPD training, the first author, an expert in SRSD research and training, delivered instruction. Prior to this session, the sixth author, a former teacher and school administrator, met with the first author for two days of training in the procedures. The first and sixth authors also developed all assessment and instructional procedures collaboratively. In addition, the sixth author attended the first PBPD session and, after modeling, practice, and feedback from the first author, delivered instruction in the remaining three states.

Critical elements of effective PBPD to support rural teachers (Borko, 2004; Desimone, 2011) and to support writing instruction (Harris et al., 2012) are italicized in the section that follows. To actively engage with colleagues of similar need and to use a team-based approach with teachers and administrators from the same school participating in PD activities, in-person PBPD training and virtual consultation were provided to each school’s general education teachers, specialists, and administrators. Training included watching and critiquing videos while also actively engaging in assessment and lesson components (e.g., assessment data collection, stages of instruction, self-regulation procedures) through modeling practice, providing opportunities for teachers to engage in active learning and collaborate with others, and providing short-term opportunities for scaffolded practice in teaching. The PBPD in-person session was subdivided into four learning phases:

A 90-min presentation on information processing, cognition, and self-regulated learning, including information on how to support student learning in each of these processes (focus on content and how students learn content; standards for writing proficiency).

A 60-min session on effective strategy instruction for writing, including the value of written expression, procedures for developing written expression, and evidence-based principles for instruction and assessment of written expression (including scoring methods for persuasive elements).

A 90-min session on SRSD for POW + TREE persuasive writing, including procedures for differentiation, for developing opportunities for scaffolded practice in the long term, for sustainability, and for in-school and district leadership.

A 60-min session for reviewing the researcher-developed writing manual that included all assessment and intervention schedules and procedures, and for addressing the feasibility of the model. In addition to the writing manual, teachers were given a short article to read (Harris et al., 2003) and the Harris et al. (2008) book, Powerful Writing Strategies for All Students. All materials from the PBPD and in the handbook were also posted on a private website for Project REAL investigators and instructors.

In-person training was not designed to stand alone, but to set the stage for virtual consultation to support long-term active engagement, teaching and learning connections, and opportunities to practice and apply instruction in the classroom. Virtual consultation was considered a critical element in reinforcing and extending what was learned during in-person training (Desimone, 2011), especially for supporting teachers’ instruction for students with disabilities (Mason et al., 2013). The sixth author provided one-on-one video-consultation via Polycom in on-demand (i.e., as teachers needed) sessions and in two 90-min mandatory video-conferencing sessions. Both mandatory sessions included a review of each lesson plan instructional step that had been completed. The first consultation session was devoted to SRSD instruction implementation and occurred during each teacher’s instructional delivery period. This first consultation focused on students’ ability in learning to use the strategy steps and the self-regulation procedures taught. For example, teachers were asked whether any student was failing to memorize the strategy steps; if so, they were provided suggestions for facilitating memorization (e.g., flashcard practice). Students’ use of self-regulation was a large focus of the virtual consultation. Teachers were asked, for example, to report interesting student self-statements and to discuss any negative self-statements needing to be addressed during instruction, and to report on the personal goals students established for their writing. The second mandatory session was a “touch-base” session prior to the first posttest. Again, the focus of this session was to ensure that all students were using the strategies and self-regulation procedures and to brainstorm solutions for any student struggling to write. In addition, procedures for collecting postinstruction writing assessments were reviewed in the second session.

SRSD instruction

Following PBPD, teachers implemented SRSD for POW + TREE as whole-class instruction. Student accommodations were provided as documented on students’ Individualized Education Program (IEP; for example, extra time, quiet table). Five 30-min lessons were taught. All six stages of strategy acquisition and four self-regulation procedures (in italics below) in the SRSD instructional delivery model were employed throughout the lessons. The POW + TREE strategy included eight elements (one topic sentence, three or more reasons, three or more explanations, and one ending sentence). See Harris et al. (2008) for full lesson plans.

Lesson 1

The students’ background knowledge for persuasive writing and strategy use, and goals for writing better papers were established in the first lesson (Establish background knowledge). The teacher reviewed key terminology (e.g., “persuasive,” “reason”), and each strategy step was reviewed and described (Discuss it). An anchor/model persuasive paper was evaluated for number of TREE elements. In closing the lesson, students were reminded of the goal: writing better persuasive papers using elements in TREE.

Lesson 2

As a warm-up activity, in this lesson and all future lessons, the students used paired practiced memorization of POW + TREE strategy steps (Memorize it). The teacher then orally read a writing prompt and cognitively modeled (i.e., modeled by thinking aloud while using self-instruction to direct writing behavior) how to use each step of POW + TREE and the self-regulation procedures for writing a persuasive paper through each writing process stage (Model it). After modeling, the teacher asked the students to write out personal self-instructions to use before, during, and after writing. Lesson 2, and all subsequent lessons, concluded with the teacher delivering verbal praise and a reminder about Memorize it practice for the next lesson.

Lesson 3

After paired Memorize it practice and discussion, student pairs collaboratively wrote a persuasive paper with POW + TREE (Support it). The students were given a blank graphic organizer, a transition word chart, their personal self-instructions sheet, and a choice of two practice prompts. Students were encouraged to use personal self-instructions throughout the writing process. After writing, the students counted and graphed the number of response elements written. A teacher could repeat the lesson if needed for individual students; no teacher reported repeating the lesson.

Lesson 4

In this lesson, the students were asked to write a persuasive paper with POW + TREE, but without instructional prompting materials. Teachers could repeat the lesson if needed for individual students; no teacher reported repeating the lesson.

Lesson 5

In this lesson, the students independently planned and composed their papers. Students also counted and graphed the number of persuasive elements written.

Instructional Treatment Fidelity

Two steps were followed to verify treatment fidelity. First, to self-monitor performance in instructional delivery, teachers were asked to use a step-by-step check sheet for each lesson step. During virtual consultation, teachers were prompted to report on their instruction by including the steps completed and on any difficulties and any successes. All teacher participants reported that lessons steps were followed and all students appeared to be learning the strategies.

Measures

Students’ written responses were used to evaluate the effects of PBPD on student outcomes. Assessment was conducted in the general education classroom; IEP accommodations were provided (e.g., extended time; quiet work space). Assessment data (student written persuasive papers) were collected prior to teacher-delivered SRSD instruction in the beginning of the school year (early fall; pretest), immediately following instruction (midfall; posttest), and in the late spring (maintenance). Training for delivering assessments was provided during in-person PBPD, and teachers were encouraged to use consultation as needed for any questions regarding procedures or accommodations. Prior to each assessment, teachers were given coded papers, that is, papers with the coded number assigned to each student. Coded papers included one set of two writing prompts, randomly assigned from a five-prompt set. This randomization was completed, so at any given measurement occasion, each class would have five prompt sets equally distributed among students. This procedure controlled for prompt effect (prior knowledge, interest, etc.) and limited potential sharing of ideas among students.

Teachers delivered all assessments by giving students their assigned paper containing the randomly assigned prompts and pencils, and by asking them to select one of the two prompts and to write a paper. As in prior SRSD research and practice, there was no time limit for writing the response (e.g., Harris et al., 2006). The number of persuasive elements was considered the measure that most represented what was taught in SRSD persuasive writing instruction. Elements were calculated with a point serial scoring system (Scardamalia, Bereiter, & Goleman, 1982) for (a) topic sentence, (b) a count for each reason, (c) a count for each explanation, and (d) ending sentence. A score of 1 was awarded if the element was present. Given that students could write multiple reasons and explanations, there was no ceiling to the elements measure.

Four advanced graduate student scorers rated all measures. Papers were coded so scorers were blind to the purpose of the study, testing time, student, teacher, school, and condition. The first and second authors conducted a 2-hr scorer-training session for scoring elements that included (a) reviewing scoring rubrics for each measure, (b) reviewing and scoring example and nonexample sample papers, and (c) providing practice in scoring example and nonexample sample papers. During scoring training, scorers rated sample papers until they achieved 95% agreement over 10 responses. Scorers then assigned scores independently. Thirty-three percent of each measure was scored twice. Reliability was established at 80% or better across each measure. Prior writing research has shown an acceptable reliability baseline range of 78% to 79% (Graham et al., 2005; Harris et al., 2006). Disagreements in two scores were averaged. Number of words, often used to measure productivity and variability in written language (Nelson, Bahr, & Van Meter, 2004), was counted by four advanced graduate student scorers. Thirty-three percent of all papers were scored twice; reliability (agreements/total) was 99%.

Analytic Steps

Our first research question was focused on significant differences in the total number of reasons and explanations between the control and intervention groups at posttest and maintenance, with additional focus on the percentage of students who included a topic sentence or ending sentence. We examined significant differences between the two groups using multivariate analysis of covariance (MANCOVA) with children nested within schools (Raudenbush & Bryk, 2002). The child’s gender (female = 0, male = 1), grade (Grade 5 = 0, Grade 6 = 1), writing ability (struggling = 0, nonstruggling = 1), and pretest scores on the variables of interest served as covariates in the analysis, with intervention status as the grouping variable (control = 0, intervention = 1). Although a Satterthwaite approximation can be used to account for unequal sample sizes, as was the case between the control and intervention groups in our study, it can also produce inaccurate p values (McDonald, 2014). Therefore, to avoid the possibility of a false positive regarding intervention effectiveness, we did not use a Satterthwaite approximation.

For our second research question regarding the total number of words and elements at posttest and maintenance, and possible differences between struggling and nonstruggling writers, we conducted multilevel model (MLM) analyses, accounting for nesting at the school level. Fixed effects included child gender, grade, writing ability, and group. Students’ pretest scores were included as a final covariate to establish a baseline for all students. Interaction effects by writing ability were estimated simultaneously with the same covariates as in the main effects models for outcomes at posttest and maintenance.

There was no missingness on any of our predictor variables; however, at posttest and maintenance, there was 6.25% missingness and 16.39% missingness on our outcome measures, respectively. We accounted for missing data using the multiple imputation (m = 20) procedure (PROC MI) in SAS 9.3. Multiple imputation is an acceptable and recommended method when data are missing at random (Schafer & Graham, 2002). All control and outcome measures were used in the multiple imputation procedure. Continuous variables were mean-centered after imputation to aid in interpretability.

For all research questions related to significant main effects and interaction effects, effect sizes for mean differences were calculated with Cohen’s d = M₁ − M₂ / σ_pooled. Effect sizes were considered to be small (.20), medium (.50), or large (.80), as suggested by Cohen (1992).

Results

Demographic and descriptive data for the 592 early adolescent students involved in this study are available in Table 1. A series of linear analyses with Tukey mean-differences comparisons and chi-square tests of independence revealed statistically significant differences between control and intervention students on some pretest measures. Although there were no significant differences in the total number of words or the inclusion of an ending sentence, students in the intervention group scored significantly higher at pretest on total elements, t(590) = −4.48, p < .001; total reasons, t(590) = −3.96, p < .001; and total explanations, t(590) = −4.54, p < .001. Students in the control group were more likely to include a topic sentence at pretest, χ²(1, N = 592) = 3.87, p = .049. Students’ pretest scores were included in analysis to account for differences at pretest. There were no significant differences based on gender, grade, or ability.

Effects for Students’ Persuasive Writing Elements

Regarding our first research question, results from MANCOVA analyses (see Table 2) suggested a significant advantage for students in the intervention group versus the control group on total reasons, total explanations, inclusion of a topic sentence, and inclusion of an ending sentence, at both posttest and maintenance, controlling for pretest performance.

Table 2.

MANCOVA Results for Writing Elements at Posttest and Maintenance.

Writing element	Posttest scores					Maintenance scores
	Control group (n = 247)		Intervention group (n = 308)		d	Control group (n = 214)		Intervention group (n = 281)		d
	M	SD	M	SD	d	M	SD	M	SD	d
Total
Reasons	2.16	1.17	2.94***	1.06	0.70	2.21	1.16	2.91***	1.14	0.61
Explanations	1.02	1.13	2.04***	1.42	0.79	0.70	1.01	1.47***	1.37	0.64
Percentage “yes”
Topic sentence	85.83	34.95	92.53*	26.33	0.21	81.31	39.08	90.39**	29.52	0.26
Ending sentence	43.32	49.65	71.43***	45.25	0.59	42.06	49.49	62.63***	48.46	0.42

Note. Bolded ds are significant effect sizes. MANCOVA = multivariate analysis of covariance.

p < .05. **p < .01. ***p < .001.

Total reasons

A significant main effect for group, while controlling for pretest, gender, grade, and ability, was found for total reasons. At posttest, F(1, 540) = 74.30, p < .001, the effect size of 0.70 favored the intervention group. At maintenance, F(1, 480) = 42.02, p < .001, the effect size of 0.61 favored the intervention group.

Total explanations

A significant main effect for group, while controlling for pretest, gender, grade, and ability, was found for total explanations. At posttest, F(1, 540) = 78.55, p < .001, the effect size of 0.79 favored the intervention group. At maintenance, F(1, 480) = 45.80, p < .001, the effect size of 0.64 favored the intervention group.

Topic sentence

A significant main effect for group, while controlling for pretest, gender, grade, and ability, was found for topic sentence. At posttest, F(1, 540) = 6.36, p = .012, the effect size of 0.21 favored the intervention group. At maintenance, F(1, 480) = 8.86, p = .003, the effect size of 0.26 favored the intervention group.

Ending sentence

A significant main effect for group, while controlling for pretest, gender, grade, and ability, was found for ending sentence. At posttest, F(1, 540) = 44.57, p < .001, the effect size of 0.59 favored the intervention group. At maintenance, F(1, 480) = 18.94, p < .001, the effect size of 0.42 favored the intervention group.

Effects on Total Words and Total Elements

Regarding our second research question, results in Table 3 suggest significant main effects with moderate to large effect sizes for group membership in favor of the intervention group on total words and total elements at both posttest and maintenance, controlling for pretest.

Table 3.

MLM Effects for Total Words and Total Elements (N = 592).

Effects/components	Posttest			Maintenance
Effects/components	B	SE	d	B	SE	d
Total words
Fixed effects
Pretest	0.37***	0.04		0.24***	0.03
Gender	−12.03***	3.17		−14.36***	3.04
Grade	14.49*	7.06		−5.14	6.71
Ability	7.42	4.24		6.69	3.96
Group	20.73*	8.12	0.47	14.96*	7.64	0.38
Variance components
Level 2 Variation	271.40**	102.56		239.51*	93.82
Level 1 Variation	1,191.36***	74.11		1,038.85***	68.19
Total elements
Pretest	0.22***	0.05		0.14**	0.05
Gender	−0.75***	0.17		−0.59***	0.16
Grade	0.07	0.39		−0.56	0.40
Ability	−0.27	0.30		0.30	0.30
Group	2.17***	0.40	0.90	1.75***	0.44	0.77
Variance components
Level 2 Variation	0.62*	0.25		0.78**	0.30
Level 1 Variation	3.80***	0.23		3.27***	0.22

Note. Bolded ds are significant effect sizes. MLM = multilevel model.

p < .05. **p < .01. ***p < .001.

Total words

At posttest, students’ pretest scores, gender, and grade were uniquely associated with their total number of words written. Students who scored higher at pretest, female students, and those in Grade 6 appeared to score higher on the total number of words written at posttest. In the presence of all covariates, our results demonstrated that students in the intervention group scored significantly higher at posttest on total words written (b = 20.73, p = .011, d = 0.47). At maintenance, students’ pretest scores and gender were uniquely associated with their total number of words written. Two groups—students who scored higher at pretest and female students—appeared to score higher on the total number of words written at maintenance. In the presence of all covariates, our results demonstrated that students in the intervention group scored significantly higher at maintenance on total words written (b = 14.96, p = .046, d = 0.38).

Total elements

At posttest, students’ pretest scores and gender were uniquely associated with their total number of elements written. Two groups—students who scored higher at pretest and female students—appeared to score higher on the total number of elements written at posttest. In the presence of all covariates, our results demonstrated that students in the intervention group scored significantly higher at posttest on total elements written (b = 2.17, p < .001, d = 0.90). At maintenance, students’ pretest scores and gender were uniquely associated with their total number of elements written. The same two groups—students who scored higher at pretest and female students—appeared to score higher on the total number of elements written at maintenance. In the presence of all covariates, our results demonstrated that students in the intervention group scored significantly higher at maintenance on total elements written (b = 1.75, p < .001, d = 0.77).

Effects by Writing Ability

Regarding our third research question, analysis of moderation effects by writing ability with respect to the total number of elements written at posttest and maintenance revealed a significant interaction at maintenance (b = 0.83, p = .045; see Table 4). The interaction between ability and intervention status is displayed in Figure 1. For struggling writers, analysis of the slope indicated the effect size (d = 0.57), although approaching significance, was ultimately not found to be significant (p = .070). For nonstruggling writers, analysis of the slope indicated the effect was significant and positive (d = 0.84, p < .001). In comparing the scores for total elements at maintenance for struggling and nonstruggling writers in the intervention group, a large effect size (d = 0.75) in favor of nonstruggling writers was detected.

Table 4.

Interaction Terms by Writing Ability for Total Elements (N = 592).

Interaction	Total elements
	Posttest		Maintenance
	B	SE	B	SE
Group by Ability	−0.07	0.47	0.83*	0.41

Note. Covariates included pretest, gender, grade, ability, and group.

p < .05.

Figure 1.

Group by Ability interaction.

Discussion

Writing is an important skill (CCSSI, 2012) and is a needed area for improvement in rural schools (National Center for Education Statistics, 2012). In 2011, only 33% of Grade 8 and 12 students in the United States were proficient in writing, whereas in rural schools, only 24% met proficiency (NAEP; National Center for Education Statistics, 2013). In the current study, an in-person and virtual consultation PBPD model for research-based writing instruction resulted in positive gains for both struggling and nonstruggling writers in Grade 5 and 6 rural classrooms. Results favored students receiving SRSD instruction compared with students in the control group, with large effects for total number of elements written (ES = 0.90) and with small effects for the number of words written (ES = 0.47). In line with prior research, students improved in writing content with greater gains when compared with word count (Graham, 2006).

As indicated by the number of elements and words written, the effectiveness of the intervention decreased over time once instruction was completed; however, significant differences maintained (ES = 0.77 for number of elements; ES = 0.38 for number of words). This result, unfortunately, is often found in intervention research and could be attenuated by booster sessions (Graham & Harris, 2003). Monitoring student progress, revisiting student goals, and supporting student writing through feedback and reinforcement over time, although noted in PBPD and the instructional manual, were not explicitly embedded in procedures. To achieve long-term and strong effects, the virtual consultation model should have included additional required sessions to improve the researcher–teacher relationship, as suggested by Knight (2011). Although research has not yet established criteria for virtual consultation following intervention implementation, at least 20 hr of contact time spread over a semester has been suggested (Desimone, 2011).

Analysis of individual components paralleled results noted in other studies with young adolescent students (e.g., Benedek-Wood et al., 2014; Mason et al., 2013.). Small but significant gains (ES = 0.21) in topic sentence writing indicated students’ skills in developing this sentence type needed improvement; less than for other persuasive elements, however. In other words, many students had skills in developing topic sentences prior to instruction; 78.22% of treatment students and 84.59% of control students wrote effective topic sentences at pretest. The large gains for writing reasons and explanations (ES = 0.70 and 0.79, respectively) and the moderate gains for writing an ending sentence (ES = 0.59) point to the areas in most need of remediation and to the strength of SRSD instruction for Grade 5 and 6 students in rural schools.

Our results also indicated significant differences in effects of SRSD instruction for struggling and nonstruggling writers. Analysis of moderation effects indicated insignificant effects for struggling learners (ES = 0.57) and significant large effects for nonstruggling learners (ES = 0.84). When comparing the two groups, the intervention was moderately more effective for nonstruggling writers (ES = 0.75). Similar differences were noted by Benedek-Wood et al. (2014), who found that following SRSD instruction, students with disabilities never reached the highest criterion level for organizational quality but 88% of responses from students without disabilities met the criterion. The differential effects based on writing ability were an important finding for two reasons. First, the positive findings for nonstruggling writers receiving SRSD instruction in whole-class instruction supports the idea that research-based instruction is beneficial for all students, not just those who struggle to learn (Marzano, Pickering, & Pollock, 2001). In addition, the smaller growth seen for struggling writers indicates a potential need for more intensive intervention, such as would be provided in Tier II or Tier III supplemental instructional time in a response to intervention (RtI) model. The gains from supplemental and additional practice, especially for struggling learners, have been well established in prior research (Harris et al., 2006; Mason et al., 2013).

Limitations

Given the large geographic area and remote location of the rural schools in our study, treatment fidelity was limited and not as rigorous as in prior SRSD studies (Mason & Graham, 2008; Mason et al., 2013). Although access to broadband Internet is limited in many rural settings (Whitacre & Mills, 2010), effective methods for observation through use of virtual audio and/or visual technology should be planned for research and PBPD consultation in rural settings.

Although PBPD did address differentiation, our results indicated students who struggle with writing need additional support, such as supplemental small-group instruction, to make gains similar to those of their peers. We encourage teachers to carefully monitor students’ written expression progress throughout instruction and provide opportunities for practice. As schools move toward RtI models for instructional delivery, our hope is that there will be more flexibility in providing all students with the time and resources needed to be successful in acquiring written expression skills. This extra time will be especially critical for students who are struggling writers, as it is clear from this study they may need extended instructional support to catch up to their peers.

Implications for Rural Classrooms

Our goal was to test the effects of SRSD for improving the persuasive writing performance of all students through whole-class instruction in the inclusive rural classroom. Both the professional development provided (i.e., 1-day workshop plus consultation) and the instructional delivery (i.e., five 45-min lessons) appear to be effective for meeting this goal. It should be noted, however, that professional development was provided to school teams, including administrative staff. Given the factors that can challenge teachers in rural schools (e.g., geographic location), it is essential that recommendations, such as team-based professional development and ongoing support, are considered (Seltzer & Himley, 1995). In other words, providing mechanisms to support teachers in SRSD writing instruction is important. In addition, researchers have documented that when implementing SRSD, it is critical, especially for students who struggle the most with writing (e.g., students with disabilities), that teachers use all stages of SRSD instruction and all procedures to support students’ self-regulation (Graham et al., 2012). Finally, information to support the implementation of SRSD writing instruction is widely available to rural educators who want to learn more and has been included in an appendix at the end of the article.

Footnotes

Appendix

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A04056. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

References

Ball

D. L.

Cohen

D. K.

(1999). Developing practice, developing practitioners: Toward a practice-based theory of professional education. In Sykes

Darling-Hammond

(Eds.), Teaching as the learning profession: Handbook of policy and practice (pp. 3–32). San Francisco, CA: Jossey-Bass.

Benedek-Wood

Mason

L. H.

Wood

Hoffman

MacGuire

(2014). SRSD for quick writing in four middle school science classrooms. Learning Disabilities: A Contemporary Journal, 12, 69–92.

Borko

(2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher, 33, 3–15.

Brendle

(2015). A survey of response to intervention team members’ effective practices in rural elementary schools. Rural Special Education Quarterly, 34, 3–8.

Cohen

(1992). A power primer. Psychological Bulletin, 112, 155–159.

Common Core State Standards Initiative. (2012). [Website]. Retrieved from http://www.corestandards.org

Cornett

Knight

(2009). Research on coaching. In Knight

(Ed.), Coaching: Approaches and perspectives (pp. 192–216). Thousand Oaks, CA: Corwin Press.

Desimone

L. M.

(2011). Outcomes: Content-focused learning improves teacher practice and student results. Journal of Staff Development, 32, 63–68.

Farmer

T. W.

Hall

C. M.

Petrin

Hamm

J. V.

Dadisman

(2010). Evaluating the impact of a multicomponent intervention model on teachers’ awareness of social networks at the beginning of middle school in rural communities. School Psychology Quarterly, 2, 94–106.

10.

Fisher

J. B.

Schumaker

J. B.

Culbertson

Deshler

D. D.

(2010). Effects of a computerized professional development program on teacher and student outcomes. Journal of Teacher Education, 61, 301–312.

11.

Flower

Hayes

(1980). The dynamics of composing: Making plans and juggling constraints. In Gregg

Steinberg

(Eds.), Cognitive processes in writing (pp. 31–50). Hillsdale, NJ: Lawrence Erlbaum.

12.

Graham

(2006). Strategy instruction and the teaching of writing. In MacArthur

Graham

Fitzgerald

(Eds.), Handbook of writing research (pp. 187–207). New York, NY: Guilford Press.

13.

Graham

Harris

K. R.

(2003). Students with learning disabilities and the process of writing: A meta-analysis of SRSD studies. In Swanson

H. L.

Harris

K. R.

Graham

(Eds.), Handbook of learning disabilities (pp. 323–344). New York, NY: Guilford Press.

14.

Graham

Harris

K. R.

Mason

L. H.

(2005). Improving the writing performance, knowledge, and self-efficacy of struggling young writers: The effects of self-regulated strategy development. Contemporary Educational Psychology, 30, 207–241.

15.

Graham

McKeown

Kiuhara

Harris

K. R.

(2012). A meta-analysis of writing instruction for students in the elementary grades. Journal of Educational Psychology, 104, 879–896.

16.

Graham

Perin

(2007). A meta-analysis of writing instruction for adolescent students. Journal of Educational Psychology, 99, 445–476.

17.

Hallahan

D. P.

Keller

C. E.

Martinez

E. A.

Byrd

E. S.

Gelman

J. A.

Fan

(2007). How variable are interstate prevalence rates of learning disabilities and other special education categories? A longitudinal comparison. Exceptional Children, 73, 136–146.

18.

Hamm

J. V.

(2017). Rural Early Adolescent Learning Program (Project REAL). Retrieved from http://soe.unc.edu/fac_research/projects/real.php

19.

Hamm

J. V.

Farmer

T. W.

Robertson

Dadisman

K. A.

Murray

Meece

J. L.

Song

S. Y.

(2010). Effects of a developmentally based intervention with teachers on Native American and white early adolescents’ schooling adjustment in rural settings. The Journal of Experimental Education, 78, 343–377.

20.

Harris

K. R.

Graham

Adkins

(2015). Practice-based professional development and self-regulated strategy development for Tier 2, at-risk writers in second grade. Contemporary Educational Psychology, 40, 5–16.

21.

Harris

K. R.

Graham

Mason

L. H.

(2003). Self-regulated strategy development in the classroom: Part of a balanced approach to writing instruction for students with disabilities. Focus on Exceptional Children, 35, 1–16.

22.

Harris

K. R.

Graham

Mason

L. H.

(2006). Improving the writing, knowledge, and motivation of struggling young writers: Effects of self-regulated strategy development with and without peer support. American Educational Research Journal, 43, 295–340.

23.

Harris

K. R.

Graham

Mason

L. H.

Friedlander

(2008). Powerful writing strategies for all students. Baltimore, MD: Paul H. Brookes.

24.

Harris

K. R.

Lane

Graham

Driscoll

Sandmel

Brindle

Schatschneider

(2012). Practice-based professional development for strategies instruction in writing: A randomized controlled study. Journal of Teacher Education, 63, 103–119.

25.

Johnson

Showalter

Klein

Lester

(2014). Why rural matters 2013-3014: The condition of rural education in the 50 states. Washington, DC: Rural and Community Trust.

26.

Knight

(2011). What good coaches do. Educational Leadership, 69, 18–22.

27.

Marzano

R. J.

Pickering

Pollock

J. E.

(2001). Classroom instruction that works: Research-based strategies for increasing student achievement. Alexandria, VA: Association for Supervision and Curriculum Development.

28.

Mason

L. H.

Graham

(2008). Writing instruction for adolescents with learning disabilities: Programs of intervention research. Learning Disabilities Research & Practice, 23, 103–112.

29.

Mason

L. H.

Kubina

R. M.

(2011). Developing writing fluency for adolescents with disabilities. In Scruggs

T. E.

Mastropieri

M. A.

(Eds.), Intervention and assessment: Advances in learning and behavioral disabilities (Vol. 2. pp. 296–319). Bingley, UK: Emerald.

30.

Mason

L. H.

Kubina

R. M.

Kostewicz

Mong Cramer

Datchuk

(2013). Improving quick writing performance of middle school struggling learners. Contemporary Educational Psychology, 38, 236–246.

31.

Mattingly

Carson

J. A.

Schaefer

(2014). Cause for optimism? Child poverty declines for the first time since before the Great Recession (Paper 221). Durham, NC: The Carsey School of Public Policy at the Scholars’ Repository.

32.

McDonald

J. H.

(2014). Handbook of biological statistics (3rd ed.). Baltimore, MD: Sparky House.

33.

McKeown

Fitzpatrick

Sandmel

(2014). SRSD in practice: Creating a professional development experience for teachers to meet the writing needs of students with EBD. Behavioral Disorders, 40, 15–25.

34.

National Center for Education Statistics. (2012). The Nation’s Report Card: Writing 2011(NCES 2012-470). Washington, DC: Institute of Education Sciences, U.S. Department of Education.

35.

National Center for Education Statistics. (2013). The Nation’s Report Card: Reading 2012 (NCES 2013-456). Washington, DC: U.S. Department of Education, Institute of Education Sciences.

36.

Nelson

N. W.

Bahr

C. M.

Van Meter

A. M.

(2004). The writing lab approach to language instruction and intervention. Baltimore, MD: Paul H. Brookes.

37.

Nippold

M. A.

Ward-Lonergan

J. M.

Fanning

J. L.

(2005). Persuasive writing in children, adolescents, and adults: A study of syntactic, semantic, and pragmatic development. Language, Speech and Hearing Services in Schools, 36, 125–138.

38.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: SAGE.

39.

Rojewski

J. W.

(1999). Occupational and educational aspirations and attainment of young adults with and without LD 2 years after high school completion. Journal of Learning Disabilities, 32, 533–552.

40.

Scardamalia

Bereiter

Goleman

(1982). The role of production factors in writing ability. In Nystrand

(Ed.), What writers know: The language, process, and structure of written discourse (pp. 175–210). San Diego, CA: Academic Press.

41.

Schafer

J. L.

Graham

J. W.

(2002). Missing data: Our view of the state of the art. Psychological Methods, 7, 147–177.

42.

Seltzer

D. A.

Himley

O. T.

(1995). Professional development and school improvement in rural schools. Journal of Research in Rural Education, 11, 36–44.

43.

U.S. Department of Education. (2016). Rural and Low-Income School Program (RLISP). Retrieved from http://www2.ed.gov/programs/reaprlisp/index.html

44.

Vygotsky

(1986). Thought and language. Cambridge, MA: MIT Press.

45.

Whitacre

B. E.

Mills

B. F.

(2010). A need for speed? Rural Internet connectivity and the no access/dial-up/high-speed decision. Applied Economics, 42, 1889–1905.