Abstract
A multiple-probe design was used to investigate planning and persuasive paragraph writing by three secondary students with disabilities and writing difficulties. The intervention consisted of one-on-one explicit instruction in persuasive writing and the use of a technology-based graphic organizer (TBGO) with embedded strategies and supports. Dependent variables included the number of sentences and overall holistic writing quality. The results were mixed. Two adolescents with the primary diagnosis of learning disabilities improved their writing performance when using the TBGO and remained above baseline levels when the TBGO was removed. One more participant diagnosed with autism demonstrated modest changes in the quantity and quality of his writing. Students reported positive perceptions of the TBGO, indicating a socially valid intervention. Implications, limitations, and directions for future research are discussed.
Keywords
Writing is an important foundational skill, vital for academic success. Students are expected to use writing to express understanding and document their learning across content areas. Writing can also enhance students’ learning (Ray et al., 2016). The skills required for effective written expression gradually develop (Graham, 2018), and educational stakeholders expect students to acquire more complex repertoires of written expression across grade levels (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010). For instance, by the end of elementary grades, it is expected for students to learn advanced sentence structure, and adolescents are expected to be able to plan, write, and revise extended compositions, such as persuasive essays (Troia & Olinghouse, 2013).
Despite written expression being a critical component of learning and expressive communication, existing data suggest that adolescents with disabilities and writing difficulties struggle with it (Graham et al., 2016). According to the National Assessment of Educational Progress (NAEP), less than a third of eighth and 12th graders are described as proficient at or at an advance level of writing (National Center for Education Statistics, 2017). Students with disabilities tend to show specific difficulties in both writing quantity and quality (Graham et al., 2016, 2017). Writing quantity includes word production and sentence fluency and writing quality involves idea generation, planning for essay parts, grammar and syntax, and holistic quality (Graham et al., 2017).
Simple View of Writing
Research on writing development suggests that writing quantity and writing quality are interrelated. Berninger and colleagues (2002) identify the functional writing system as a complex system with many factors influencing text generation at the word, sentence, and discourse levels. This simple view of writing begins with transcription skills. Transcription skills are the foundational skills of writing, just as decoding skills are the foundational skills of reading. Students with various disabilities and writing difficulties often struggle with transcription skills (Bouck et al., 2010; Graham et al., 2017; Park et al., 2017). Handwriting is often illegible and typing is often slow and labored. Transcription skills affect the quantity of student writing.
Another feature of the simple view of writing is the executive functions, including the ability to maintain attention, set goals, plan, review, revise, and use strategies for self-monitoring and self-regulation. Working memory helps coordinate writing task demands and retrieves from long-term memory previously learned information such as vocabulary and text structures, or even personal experiences to come up with ideas for writing (Graham, 2018). The metacognitive demands underlying the writing process, including the efficient use of working memory and self-regulation tasks, may be difficult for students with disabilities (Hacker, 2018). They may struggle to come up with ideas, organize those ideas, and translate them into meaningful text (Arabsolghar & Elkins, 2000). Executive functions affect the quality of student writing. Without intensive instruction to support the quantity and quality of student writing, adolescents with disabilities and writing difficulties are unlikely to become proficient writers.
Intensive Writing Instruction for Adolescents With Disabilities
Over time, the quantity and quality of writing proficiency are compounded, and teachers need to first improve secondary writing instruction if students are to meet the common core standards (Graham et al., 2014; Kiuhara et al., 2009; Sundeen, 2015). As the academic demands increase from elementary school, surveys of middle school and high school teachers indicate that very little time is spent teaching students to write, and adaptations or evidence-based practices for writing are used infrequently (Graham et al., 2014; Kiuhara et al., 2009). Adolescents with varying disabilities, including specific learning disabilities (SLDs), and those with limited cognitive ability as indicated by standardized achievement tests may require intensive writing instruction. This type of small-group or 1:1 instruction is characterized by evidence-based intervention implementation, more practice opportunities, and immediate corrective feedback (Fuchs et al., 2014). For secondary (ages 13–18) students with disabilities who require intensive writing instruction, most relevant interventions to date incorporate some type of strategy instruction, including the process/writing workshop approach (Liberty & Fitzpatrick, 1994), modified self-regulated strategy development (SRSD) instruction (Guzel-Ozmen, 2006; Konrad et al., 2017; Konrad & Test, 2007), graphic organizer instruction (Park et al., 2017), and specific software tools (Bouck et al., 2010). In these identified studies, participant performance increased across relevant dependent measures, including writing quality, following teacher-led intensive intervention (e.g., across high dosage of intervention; Konrad et al., 2017; Konrad & Test, 2007), and/or technological support, especially in the area of planning and brainstorming (Bouck et al., 2010).
Intervention Framework: Technology-Based Writing Instruction
Meta-analytic research has provided evidence that technology can be beneficial for students when completing writing tasks, regardless of disability (Morphy & Graham, 2012). Despite this fact, there continues to be a lack of technology incorporated into writing instruction and limited research regarding the efficacy of technology-based interventions that could support students with disabilities and writing difficulties (Evmenova & Regan, 2019). One promising way to integrate technology into writing instruction is to use a technology-based graphic organizer (TBGO) to help students visually organize their ideas for writing. Several benefits to using a TBGO include increased legibility and the ability to manipulate and revise more readily and easily. Ciullo and Reutebuch (2013) conducted a literature review and located 12 studies that incorporated a TBGO into instruction for students with learning disabilities (LDs), with promising evidence for written expression. In addition, two studies used technology for the planning process and showed promise for students with other disabilities (Bouck et al., 2010; Park et al., 2017).
Several recent studies have used a TBGO with embedded self-regulated learning strategies to improve the persuasive writing of students with various abilities and needs (e.g., Evmenova et al., 2016, 2020; Regan et al., 2018). All studies have found that after four 50-min lessons and sufficient opportunities to practice using the TBGO, students with and without disabilities improved their quality of writing, whereas students with disabilities also increased the quantity of their writing. Specifically, in one study, 10 seventh- to eighth-grade students with high-incidence disabilities (i.e., LDs, emotional and behavioral disorders, attention-deficit/hyperactivity disorder, and autism spectrum disorder) increased the number of words, sentences, and transition words written as well as included more essay parts and received a higher holistic writing quality score when introduced to the TBGO (Evmenova et al., 2016). Similarly, students with and without disabilities in inclusive seventh-grade classrooms who used the TBGO outperformed students in the control group who received traditional writing instruction on measures of number of transition words and holistic writing quality (Regan et al., 2018). These results further generalized to the inclusive upper elementary social studies and science classrooms. In another study, students produced longer essays with more transition words and demonstrated higher scores on essay parts measure when writing with the TBGO (Evmenova et al., 2020). Across all studies with the TBGO, students were also able to maintain these gains after the TBGO was removed. In addition, students’ attitudes toward writing with the tool were very positive. However, none of these studies included adolescents with disabilities and writing difficulties in high school.
Current Study
Due to the multifaceted, persistent academic deficits exhibited by adolescents with disabilities, it is critical to examine ways in which educators can support the unique needs of these students, particularly in the area of writing. The purpose of this study is to examine the functional relation between using a TBGO with embedded strategies and brainstorming supports and improvements in persuasive writing by high school students with disabilities and writing difficulties. Specific research questions included the following:
Method
A multiple-probe across participants (Tawney & Gast, 1984) single-subject design was used to assess the changes in students’ writing performance when using a TBGO with embedded strategies and brainstorming supports. Baseline consisted of three to five probes to establish typical persuasive writing performance of the students and to document student need for a writing intervention. One-to-one instruction was introduced in a staggered fashion. A functional relation was demonstrated only if student participants’ writing improved after instruction and while other participants still in baseline stayed at or near preintervention levels. Intermittent, but consistent, data collection in baseline limited potential student frustration with a writing task. Probes were collected at strategic points: before and after the independent variable was introduced in each tier, allowing for overlap of data across phases (Kratochwill et al., 2013).
Setting
The study took place in a large, diverse school district located in the mid-Atlantic region of the United States. Specifically, all participants were enrolled in a large, high-needs public secondary school serving approximately 3,000 students in Grades 7 to 12. At the time of the study, 68% of all enrolled students represented minority populations and 30% of students came from economically disadvantaged backgrounds. All assessments and instruction in this study took place during the 90-min literacy elective course block. Baseline, TBGO use, and maintenance probes as well as all instructional sessions were conducted in a teacher’s lounge room. The room was roughly 25 × 25 feet and contained a large table with chairs around it. The first author instructed students in a one-on-one setting.
Participants
After all necessary permissions were obtained from the university and school-level Institutional Review Boards, the recruitment of participants took place. Three students met the following inclusion criteria: (a) between 14 and 19 years of age; (b) a documented diagnosis of any of the following disabilities or combination of disabilities, including but not limited to SLDs, mild intellectual disability (ID), autism, and/or attention-deficit/hyperactivity disorder (ADHD); (c) intelligence quotient (IQ) in the range of 60 to 70 (±5); (d) writing goal present on the Individualized Education Program (IEP); (e) standardized reading and writing scores below the 10th percentile compared with grade-age peers (scores < 80); (f) working memory significantly impacted (Working Memory Index [WMI] < 80); (g) deficits in receptive and/or expressive language as evidenced by a speech language goal on the IEP; (h) ability to independently type at least 12 correct words per minute (CWPM); (i) ability to compose a complete sentence; and (j) accessing general education standards and state assessments. Demographic information was collected during the initial screening procedures as described below. Participants were 16-year-old 10th graders who received instruction across all academic areas in a small-group, self-contained setting. All participants were able to independently complete the tasks on the technology-based assessment and typed more than 12 CWPM. Overall demographics can be seen in Table 1.
Participants’ Demographic Characteristics.
Note. SLD = specific learning disability; ADHD = attention-deficit/hyperactivity disorder; WMI = Working Memory Index; KTEA = Kauffman Test of Educational Achievement; WE = written expression; RC = reading comprehension; LC = listening comprehension; CWPM = correct number of words per minute on the typing test.
Janet
Janet was a biracial (Asian and African American) female diagnosed with SLD and ADHD. Janet received speech and language therapy once a week. Her WMI and Kaufman Test of Educational Achievement–Third Edition (KTEA-3) scores indicated significant needs in the areas of written expression, reading, and listening comprehension. In the area of writing, Janet struggled with idea generation and initiating writing tasks, along with difficulty organizing her thoughts even with the aid of a graphic organizer. She had a writing goal on her IEP to use a prewriting strategy, such as a graphic organizer, to write one to three well-organized paragraphs with a minimum of five sentences per paragraph. Her accommodations included access to a word processor with a spell checker and access to graphic organizers for writing assessments.
Trent
Trent was an African American male diagnosed with SLD and ADHD. Trent received speech/language therapy and occupational therapy services once a week. His WMI and KTEA-3 scores indicated significant needs in written expression, reading, and listening comprehension. In the area of writing, Trent was able to come up with very creative ideas although they were not always on topic. He began writing tasks easily but spent very little time planning and writing, which resulted in very simple sentences that were often disorganized. His writing goal on the IEP was to use a prewriting strategy, such as a graphic organizer, to write one to three well-organized paragraphs. His accommodations included access to a word processor with a spell checker and access to graphic organizers for writing assessments.
Tom
Tom was an African American male diagnosed with autism. Tom received speech and language therapy services once a week. His WMI and KTEA-3 scores indicated significant needs in written expression, reading, and listening comprehension. In the area of writing, Tom struggled with idea generation and organization, even with the aid of a graphic organizer. His sentences tended to be short and simplistic and he tried to complete tasks as quickly as possible as writing was not a preferred activity. His writing goal on the IEP was to use a prewriting strategy, such as a graphic organizer to write one well-organized paragraph with at least five sentences. His accommodations included access to a word processor with a spell checker and access to graphic organizers for writing assessments.
Independent Variable and Materials
The intervention in this study included two major components: (a) a TBGO with embedded strategies and brainstorming supports, and (b) explicit instruction of the persuasive genre and the use of the TBGO, which is described in more detail under “Procedures” section.
The TBGO was created as a Chrome application and included five major parts: (a) choosing a topic/goal setting, (b) brainstorming, (c) translating ideas into sentences using a table, (d) self-monitoring, and (e) self-evaluating (see website https://wego.gmu.edu). First, after selecting one of the prompts to write about, students chose a goal from a drop-down menu setting up a clear and attainable end point for their writing (e.g., I will include three reasons and one example; I will include three reasons and two examples; or I will include three reasons and three examples).
In the second part of the TBGO, students started by brainstorming what they wanted to write about their chosen topic. In the current study, the brainstorming space was modified from previous research (see Evmenova et al., 2016). The brainstorming area was enhanced with a color-coded web that included prompts for all major parts of a persuasive essay (see website https://wego.gmu.edu/tbgos.html). Opinion and summary parts were orange, the first reason and example were blue, the second reason and example were green, and the third reason and example were yellow. Students were asked to write down key words within the color-coded web and then transfer their ideas into the similarly color-coded sections in Part 3 of the TBGO.
In Part 3, the far-left column of the TBGO had an IDEAS mnemonic to help students remember the required parts of a persuasive paragraph (i.e., I = Identify your opinion, D = Determine three reasons, E = Elaborate with examples, A = Add transition words, and S = Summarize). Visual cues, text prompts, audio comments, and examples were embedded as additional supports to be accessed as needed. Misspelled words were highlighted and text-to-speech was available. After students moved their ideas from the brainstorming box, they could write complete sentences choosing various transition words from the drop-down menu (e.g., “First,” “For example”). In the very right column of Part 3 in the TBGO, the self-monitoring statements were located for students to check off whether the statements were true. If any of the boxes were left unchecked, students were unable to move to the next part of the TBGO.
After completing self-monitoring, students clicked a “copy” button to move to the next page of the TBGO for Part 4. Here, all written sentences were automatically copied into a paragraph format. After students listened to their final written product using text-to-speech feature, students were able to make any necessary revisions. Finally, the final part (Part 5) of the TBGO, referred to as self-evaluation, was completed by answering questions about their writing (e.g., How many reasons did I include in my essay?). The completed TBGOs and student writings were emailed to the researcher, teacher, and the students.
Other materials used in this study included (a) writing prompts validated by previous research (e.g., Evmenova et al., 2016); (b) a researcher binder, including lesson plans and activities for explicit instruction; (c) student folders, including an IDEAS strategy card and an agenda for each lesson; and (d) technology (i.e., Dell laptops with Chrome browser, camera, audio recorder for backup).
Dependent Measures
Two dependent variables reported here were the number of sentences and holistic quality of writing. A group of words was considered to be a sentence if it contained a noun and a verb and represented a complete thought. Sentences did not have to start with a capital letter but had to include ending punctuation to be considered a sentence. The number of sentences was manually counted in each essay. A holistic quality rubric, modified from former research (Evmenova et al., 2016), was used (ranging from 1 point to 8 points) to determine overall quality of the response. Criteria per point value was determined by the number of essay parts included, use of transition words, and how well the overall product was written. For example, a score of 0 was given if there were no essay parts written in complete sentences. A score of 8 was given if it included a discrete topic sentence, three discrete reasons, at least two discrete examples, a discrete summary, at least two transition words used correctly, and a paragraph written in a logical sequence that strengthened the writer’s argument (see Evmenova et al., 2016, for detailed description). Each essay was read at least twice before scored by the first author.
Probes
A pool of 26 prompts was created from published practice standardized assessments of persuasive writing and validated by previous research (e.g., Evmenova et al., 2016). The first author, in collaboration with the teacher, reviewed all prompts for interest, readability, and cultural appropriateness prior to the study. Sample prompts included, “Write an essay on whether or not students should go to school on Saturday,” or “Write an essay on whether or not students your age should do chores at home.” In each session throughout the study, two prompts were randomly selected from the pool. Both prompts were presented on paper and read aloud by the researcher. Students were asked to choose one prompt to write about.
Interobserver Agreement
Two independent observers were trained to score writing samples until full agreement was met during training. After training, one observer scored 100% of written paragraphs (39 total samples) across all students and all phases. Interobserver agreement for both dependent variables was calculated by dividing the number of agreements by agreements plus disagreements. Disagreements were discussed until 100% agreement was reached. Then, the second observer scored 33% of writing samples across students and phases. Using the same formula, 100% agreement was reached for both dependent variables.
Procedures
The procedures in this study were as follows.
Initial screening
To determine whether potential participants met the inclusion criteria, the initial screening was conducted. Students’ IEP files were carefully reviewed to collect the most recent data on their ability and performance (e.g., IQ, WMI; scores on the KTEA-3). All aforementioned tests were administered to students by school personnel prior to the study and were reported in the files. In addition, the first author consulted with the teacher and observed the potential participants during classroom activities to determine their ability to compose complete sentences. Finally, an initial interview with each participant was conducted. The first author asked students 1:1 to share their feelings about writing. During the interview, a technology-based assessment was also implemented. The assessment was developed by the researchers and validated by previous research (e.g., Evmenova et al., 2016). Participants were asked to perform several tasks on the computer that would be required while using the TBGO (e.g., opening a website, using a drop-down menu, scrolling). At the end of the screening, students were asked to type a 1-min passage to determine typing rate (reported in Table 1).
Baseline
Baseline consisted of three probes in Tier 1 (Janet), four probes in Tier 2 (Trent), and five probes in Tier 3 (Tom) to establish typical persuasive writing performance of the participants and to document the need for a writing intervention. Individually, students were pulled from their 90-min literacy elective course block and asked to write in response to one of two prompts on a computer using Microsoft Word. Traditional instruction during the elective course offered remedial instruction in reading and writing, including how to write and edit a persuasive essay. During baseline, participants had access to a paper-based graphic organizer that looked like the color-coded web within the TBGO brainstorming section. A standardized script was used to provide directions. Those included (a) asking students to choose one prompt to write about (prompts were provided in the Word document and on top of the paper-based graphic organizer in the Baseline phase); (b) encouraging students to do their best and write independently; and (c) stating that the researcher could only offer technical help and could not answer any questions about writing. Students were asked to raise their hand when they were finished, so that the researcher could ensure that the file was saved correctly. Finally, the researcher read both prompts out loud and asked students to begin. No talking to each other or taking breaks (leaving the area) were allowed during the session. Each session was untimed and students could take as much or as little time as they needed.
Instructional procedures and TBGO use
Then, students were introduced to the TBGO using the lessons plans designed to include scaffolds, repetition, and sufficient practice opportunities. Lesson 1 focused on learning about the persuasive genre and exploring the IDEAS strategy. Students watched several video clips to enhance their understanding of persuasion and to collaboratively label the parts of two example persuasive paragraphs. Lesson 2 provided additional practice learning and applying the IDEAS strategy to label paragraph parts. A think-aloud process was used to model how students should evaluate the two prompts provided and how to choose one that they have more to say about. Students were introduced to the process of brainstorming and the color-coded web within the TBGO. In Lesson 3, students practiced completing the TBGO along with the teacher. Explicit instruction on self-regulated learning strategies and built-in supports followed. Lesson 4 included the researcher modeling both the thinking and writing process while completing the TBGO from the beginning to end. The IDEAS mnemonic, self-regulated learning strategies, and built-in supports were once again reviewed and reinforced. Students then transferred one of their baseline writing samples into the TBGO. Missing parts were discussed and potential elaborations were identified. Lesson 5 included independent use of the TBGO. A mastery criterion checklist was used by the primary researcher to determine whether students were ready to independently complete the TBGO. Once students met the criteria for independence, they moved to the TBGO use phase while other students engaged in additional practice with the TBGO under the researcher’s supervision.
Lessons were delivered in a one-on-one setting. On average, lessons lasted 37 (range = 18–51) min. Janet received 3 hr 25 min of instruction (average time per lesson: M = 41 min; SD = 10.22). Trent received 2 hr 58 min of instruction (average time per lesson: M = 36 min; SD = 8.93). Tom received 2 hr 23 min of instruction (average time per lesson: M = 34 min; SD = 11.72). Following instruction, each student entered the independent TBGO use phase in a staggered fashion. Tier 1 (Janet) entered this phase during Session 9, Tier 2 (Trent)—during Session 15, and Tier 3 (Tom)—during Session 21.
During TBGO use phase, students were asked to write in response to one of two prompts using the TBGO. Just like in baseline, they had unlimited time to write. The researcher followed a similar script to provide standardized directions of (a) asking students to choose one prompt to write about (prompts were provided on a piece of paper in the TBGO use phase); (b) encouraging students to do their best and write independently; and (c) stating that the researcher could only offer technical help and could not answer any questions about writing. Students were asked to raise their hand when finished, so that the researcher could ensure that the file was saved correctly. Finally, the researcher asked students to open the TBGO website and type in their name and the date. She then read both prompts out loud and asked students to begin. No talking to each other and taking breaks (leaving the area) were allowed during the session.
Maintenance
Two weeks after the last TBGO use session, the first author returned to the classroom to examine whether students maintained their writing performance after TBGO withdrawal. Similar to previous research (e.g., Evmenova et al., 2016), one lesson was delivered in a small group prior to collecting maintenance data. That lesson focused on modeling how to write without the TBGO using learned strategies and skills. Students were given a paper-based graphic organizer like the one used in baseline and the first author modeled how to use it by following a think-aloud protocol. After the small group lesson, students participated in 3 one-on-one maintenance sessions when they were asked to write in response to one of two prompts using Microsoft Word and a paper-based graphic organizer. The first author followed the same script as used in the baseline phase to provide standardized directions that included reading both prompts out loud and not providing any assistance. Students had an unlimited time to write.
Fidelity of Implementation
All testing and instructional sessions were either audio or video recorded. Checklists were created to determine the procedural reliability for each phase of the study. A special education teacher unfamiliar with the purpose of the study watched 100% of recordings (37 video files and 18 audio files). Procedural reliability was established by dividing the number of steps completed by the number of steps planned, resulting in 100% across all phases.
Social Validity
One-on-one semi-structured interviews were conducted with students before and after the study to assess the social validity of the intervention and answer the third research question. The preintervention interview included eight questions (e.g., Do you like to write? Do you prefer to write by hand or on the computer?), whereas the post-interview included 23 questions (e.g., What did you think about the graphic organizer? What does IDEAS stand for?). The interview questions were developed by the researchers and validated in previous studies across grade levels (e.g., Evmenova et al., 2016; Regan et al., 2018). The readability of interview questions was slightly adapted in collaboration with the teacher to make sure these students fully understood what was being asked. Prompting was used throughout to encourage students to respond (e.g., Tell me why). Interviews were audio recorded and transcribed verbatim.
Results
The data were analyzed using visual inspection across baseline, TBGO use, and maintenance phases (Kratochwill et al., 2013). Visual analysis focused on changes in level, trend, variability, immediacy of effects, overlap, and consistency. Tau-U was also calculated for each dependent variable and each participant (Parker & Vannest, 2014; http://www.singlecaseresearch.org/calculators/tau-u).
TBGO Use
Overall, the results were positive but mixed. As can be seen in Figures 1 and 2, Janet and Trent increased the quantity (the number of sentences) and quality (holistic writing quality) of their writing, whereas Tom’s changes were modest. Janet and Trent demonstrated 100% PAND for both dependent variables indicating high effectiveness of the intervention. The Tau-U for all participants combined was 0.97 (p = .00), 90% confidence interval (CI) = [.60, 1] for the number of sentences indicating a large effect, and 0.89 (p = .0001), 90% CI = [.51, 1] for holistic writing quality indicating a medium effect.
Total number of sentences written across baseline, TBGO use, and maintenance phases.
Holistic writing quality data across baseline, TBGO use, and maintenance phases.
Janet
As can be seen in Figure 1, Janet’s number of sentences in baseline was stable, averaging 1.33 (SD = 0.57). There was a downward trend and low variability of number of sentences data. Following instruction, TBGO use data showed an immediate change in level to an average of 6.57 (SD = 0.79) sentences with a slightly downward trend and medium variability of data. There was no overlap between baseline and TBGO use data, with Tau-U = 1 (p = .02), 90% CI = [.31, 1] demonstrating a large effect. As can be seen in Figure 2, Janet’s holistic quality data in baseline were stable, averaging at 1.33 (SD = 0.58). There was a downward trend and low variability of her writing quality data in baseline. Following instruction, TBGO use data showed an immediate change in level to an average writing quality of 7.57 (SD = 0.79) with a slightly downward trend and medium variability of data. There was no overlap between baseline and TBGO use data, with Tau-U = 1 (p = .02), 90% CI = [.31, 1] demonstrating a large effect.
Trent
Trent’s number of sentences in baseline was stable, averaging at 1.75 (SD = 0.50). There was a flat trend and low variability of data. Following instruction, TBGO use data showed an immediate change in level to an average 7.00 (SD = 0.79) sentences with a slightly downward trend and medium variability of data. There was no overlap between baseline and TBGO use, with Tau-U = 1 (p = .01), 90% CI = [.36, 1] demonstrating a large effect (see Figure 1). Trent’s holistic writing quality data in baseline were stable, averaging at 1.50 (SD = 0.58). There was a flat trend and low variability of data. Following instruction, TBGO use data showed an immediate change in level to an average 5.00 (SD = 2.45) with a downward trend and high variability of data. There was no overlap between baseline and TBGO use, with Tau-U = 1 (p = .01), 90% CI = [.36, 1] demonstrating a large effect (see Figure 2).
Tom
Tom’s number of sentences in baseline was highly variable, with a flat trend, averaging at 2.6 (SD = 1.34). Following instruction, TBGO use data showed an immediate change in level with a slightly downward trend and medium variability of number of sentences data averaging at 5.60 (SD = 0.89). There was some overlap between baseline and TBGO use data, with Tau-U = 0.92 (p = .02), 90% CIs = [.29, 1] demonstrating a large effect. Tom’s holistic writing quality data in baseline were highly variable, with flat trend, averaging at 2 (SD = 1.41). Following instruction, TBGO use data showed no immediate change in level, but an upward trend and low variability of writing quality data averaging at 3.80 (SD = 0.84). There was some overlap between baseline and TBGO use data, with Tau-U = 0.68 (p = .08), 90% CI = [.05, 1] demonstrating a medium effect.
Maintenance
Two weeks after the last TBGO use probe, the maintenance lesson and then maintenance phase took place. Students wrote in response to a prompt using Microsoft Word and a paper-based graphic organizer. As can be seen in Figures 1 and 2, Janet and Trent maintained their performance above baseline levels for the number of sentences (M = 6.00; SD = 1.00 and M = 4.33; SD = 0.57, respectively) and holistic writing quality (M = 5.33; SD = 1.15 and M = 4.33; SD = 0.58, respectively). Tom did not write more in maintenance than in baseline (number of sentences: M = 4.00; SD = 0.00). In terms of holistic writing quality, he scored 4.00 (SD = 0.00) in all maintenance probes, demonstrating improved consistency of his writing performance.
Social Validity
Both pre- and postintervention interviews were transcribed verbatim by the first author and analyzed using thematic analysis. Specifically, transcripts were read and reread to identify any emerging and common codes that were later grouped into overarching themes. It should be noted that all students struggled with expressive language and received speech/language services weekly. At the end of the study, all participants were able to recall the IDEAS strategy and correctly explain each letter of the mnemonic. All students reported that the goal setting section was important because it helped “. . . know how many sentences I needed to have” and “keeps me from writing slow.” Trent was the only student who reported using the text hints and audio comments embedded in the TBGO, and he found them helpful. All students really enjoyed the transition words drop-down menu and were able to name several transition words that they remembered from their work in the study. They also appreciated the feature in the TBGO that automatically turned their sentences into a paragraph, stating it made things “easier and faster.” Tom suggested that the TBGO should offer word prediction supports. Janet and Tom said that they had never used a tool like this one prior to the study, whereas Trent remembered that he had used a graphic organizer before but “. . . never a graphic organizer on the computer.”
All students liked the lessons, and Janet reported that they were “kind of easy.” When asked whether they preferred the 1:1 or small group instruction with the first author, all students said that they preferred the 1:1 instructional model, stating it was “easier to concentrate” and had “less distractions.” Overall, Trent and Tom reported that they liked using the TBGO because it “makes writing easier” and “we can think of more ideas.” Both students also said that they thought their writing improved as a result of using the IDEAS strategy and the TBGO. On the contrary, Janet was not sure how much she liked the TBGO. She “sort of” liked it, but using it and writing in general was still “hard” because “it is hard to think of something to write about.”
Discussion
The purpose of this study was to examine a functional relation between using a TBGO with embedded strategies and brainstorming supports and improvements in persuasive writing by high school students with disabilities who required intensive writing instruction. Analysis of data across participants and phases demonstrated positive but mixed results. Two students with the primary diagnosis of LDs, Janet and Trent, were able to increase the number of sentences and, most important, improve the overall holistic quality of their writing when using the TBGO. Writing a well-composed six- to eight-sentence paragraph was an improvement over the one-to-two short simple sentences produced by the two 10th graders during baseline. Unlike paragraphs composed in the baseline phase, Janet and Trent included reasons, some examples, and a summary of their opinion when using the TBGO. Furthermore, these two students were able to maintain the quantity of their writing above baseline levels when the TBGO was removed. At the same time, a student diagnosed with autism, Tom, demonstrated modest changes in his writing.
These findings are somewhat consistent with previous research on the use of the TBGO by students with various abilities and needs (e.g., Evmenova et al., 2016, 2020; Regan et al., 2018). This was the first known study in which a TBGO with embedded strategies and supports was used with high school students who require intensive instruction for improving writing performance. Also, in contrast to the previous TBGO-related studies, participants in this study had standardized testing scores that indicated cognitive ability that is typically associated with students with a mild ID. The majority of research on writing with technology to this point has included students with LDs or emotional and behavior disorders (e.g., Evmenova et al., 2020; Regan et al., 2018) or students with an ID (e.g., Bouck et al., 2010; Park et al., 2017). Although there were three participants with the primary diagnosis of autism in Evmenova et al.’s (2016) study who showed great improvements in the quantity and quality of their writing following the use of the TBGO, they had substantially higher IQ levels (126, 114, 98) than Tom (66). Although the delivery of intensive instruction (i.e., 1:1, ample practice, corrective feedback) in this study allowed Tom to write slightly better when he used the TBGO, future research should explore how to further individualize instruction, especially for adolescents with autism who present with an IQ in the extremely low range.
For example, additional attention should be paid to the dosage of intervention when working with students who need intensive instruction (Lemons et al., 2018). Whereas decreasing group size was not an issue in this study as instruction was 1:1, increasing the frequency and duration of the intervention may be needed. In previous writing research, the number of instructional sessions ranged from 10 to 19 prior to moving into independent writing phase (e.g., Konrad et al., 2017; Konrad & Test, 2007). While that was not possible in the current study due to school district parameters, further exploration of how much instruction and practice is needed to further improve writing outcomes for all students is warranted. Future researchers may want to assess how different dosage may influence students’ writing performance and its maintenance. In addition, the National Center on Intensive Instruction (NCII; https://intensiveintervention.org) suggests that individualization for those students who require intensive instruction is a process that relies heavily on teachers collecting student data, analyzing the data based on decision rules, and making modifications to instruction, as needed. When employed in the classroom, frequent monitoring of students’ writing progress with the use of the TBGO would allow for precise individualization (Regan et al., 2021).
In the current study, students were given access to a paper-based graphic organizer during baseline. Despite the fact that all student participants had a graphic organizer listed on the IEP as a regular accommodation, all students failed to use it for planning their writing in baseline. Following explicit instruction, modeling by the first author, and the use of the TBGO, all students consistently took advantage of using the paper-based graphic organizer during the maintenance phase. The improvements in writing for two students at maintenance are consistent with previous research that suggests spending time on planning and organizing thoughts (either using a technology-based or a paper-based graphic organizer) may be a critical component of the writing process (Ciullo & Reutebuch, 2013; Guzel-Ozmen, 2006). These results suggest that teachers should not only increase the frequency with which they provide explicit writing instruction but should also employ strategies such as a graphic organizer to improve their writing instruction for adolescents with disabilities who struggle with writing (Kiuhara et al., 2009). This is especially critical to ensure that students with disabilities are able to meet the common core state standards (Sundeen, 2015).
Practical Implications
As a writing intervention package, the use of the scripted lesson plans and TBGO was a feasible and cost-effective option. This writing intervention could be successfully implemented across a variety of settings, age categories, grade levels, and disability groups, including students with various writing abilities and learning needs. Although, in the current study, the intervention was implemented by the first author in one-on-one settings, previous research has shown that class-wide implementation by teachers in an authentic school setting is possible (e.g., Evmenova et al., 2020; Regan et al., 2018).
There are several instructional recommendations based on the results of this study. First, teachers should provide more extensive practice using the TBGO and provide immediate direct feedback on writing samples produced with the TBGO. Regular, short conferencing sessions to field questions should be completed and positive reinforcement should be provided to students while they are independently writing to encourage student engagement in the writing process. Foundational writing skills need to be addressed prior to focusing on paragraph writing. A prerequisite skill should include writing complete sentences consistently. As the next step, specific instruction in the editing process could be added to improve the overall holistic quality of student writing. Finally, teachers should remember to encourage students to set high goals for their writing and continue to use the built-in scaffolds and supports (Evmenova & Regan, 2019).
Limitations and Future Research
These findings need to be considered with caution due to the following limitations. While a small number of participants is common for single-subject studies, replicating a study with more homogeneous participants is needed. Future studies, especially inclusive of students with autism, will provide a better sense of who can benefit from the TBGO intervention. The block schedule used in the students’ school did not allow flexibility in the length of each instructional session. The researchers were allowed to work with each student for no longer than 45 min to allow students to participate in other classroom activities for the remainder of the block time. Indeed, the overall instructional time in this study was relatively low compared with other writing studies (e.g., Guzel-Ozmen, 2006; Konrad et al., 2017; Konrad & Test, 2007). More research with teachers as implementers is encouraged to examine the use of TBGO in authentic settings.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This research was produced under the U.S. Department of Education, Office of Special Education Programs No. H327S120011. The views expressed herein do not necessarily represent the positions or policies of the Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service, or enterprise mentioned in this publication is intended or should be inferred.