Abstract
The role of occupational therapy practitioners in the school system is evolving. The Individuals With Disabilities Education Improvement Act of 2004 (IDEA; Pub. L. 108–446) broadened the scope of the original legislation (Individuals With Disabilities Education Act of 1990; Pub. L. 101–476) to focus on improving student outcomes, preventing problems, and setting expectations for students with disabilities to achieve high standards (Jackson, 2007). Preventive strategies in the school systems are addressed through mechanisms such as early intervening services (EIS) and response to intervention (RtI; American Occupational Therapy Association [AOTA], 2011; Jackson, 2007). IDEA allows occupational therapy practitioners to consult with and sometimes provide direct services for students in general education, especially for students struggling with learning or behavior. This expanded role for practitioners fosters both a consultative and collaborative environment between practitioners and teachers (AOTA, 2011).
Occupational Therapy in General Education
Within general education, occupational therapy practitioners may be involved in EIS and RtI. EIS is a provision that allows schools to use some of their federal IDEA funding to provide training for teachers or to fund services for general education students. For example, practitioners may provide consultation regarding education concerns observed in general education students or may be asked to recommend a multisensory handwriting instruction approach to implement schoolwide (Jackson, 2007).
RtI, an example of EIS, is a process that monitors the success of instructional strategies and services that are being implemented individually, in small groups, or classwide (Jackson, 2007). RtI, although implemented differently by state, is typically a three-tiered model in which the first and foundational tier is focused on the effectiveness of education for all students, thus requiring evidence-based curricula and high-quality instruction (AOTA, 2014). Through the RtI model, occupational therapy practitioners may be involved at any level of implementation (AOTA, 2012). Because the importance of handwriting instruction and handwriting skills continues to be documented (Puranik & Alotaiba, 2012), it is an important area of concern for school teachers and administrators that is often addressed by occupational therapy (Asher, 2006; Case-Smith, 2002; Hoy, Egan, & Feder, 2011). Examples of the practitioner’s involvement in general education at Tier 1 include providing training for school personnel on handwriting strategies, assisting with handwriting screening, and suggesting research-based handwriting curricula (AOTA, 2011, 2012, 2014).
When exploring current advancements in the roles of occupational therapy within an RtI model, a recent study of 276 school-based occupational therapists indicated that more than half had been involved in problem-solving teams, coaching and consultation, and one-on-one intervention. Other methods of involvement in RtI included identification of students needing extra support, in-services, progress monitoring, data collection, universal screening, program and curriculum development, and leadership or coleadership of groups (Cahill, McGuire, Krumdick, & Lee, 2014). Occupational therapy practitioners also advocate for occupational therapy services to enhance student education under IDEA provisions (AOTA, 2011).
Evidence-Based Practice in General Education
The implementation of EIS and RtI requires evidence to be used in decision making regarding educational practices and curriculum selections. Therefore, successful steps have been taken to ensure that school-based occupational therapy practitioners are equipped to use evidence-based practice in the school setting (Cahill, Egan, Wallingford, Huber-Lee, & Dess-McGuire, 2015). Recent research includes studies that have involved the implementation of occupational therapy–based approaches aligning with RtI in general education. Although this area of research is in its infancy, studies have been published supporting the consultative and collaborative efforts of practitioners and teachers to address student skills in general education (Howe, Roston, Sheu, & Hinojosa, 2013; Ohl et al., 2013). In addition, the Write Start program is a recent example of a coteaching model for handwriting skills involving a general education teacher and an occupational therapy practitioner. Multiple studies have documented the effectiveness of this model (Case-Smith, Holland, & Bishop, 2011; Case-Smith, Holland, Lane, & White, 2012; Case-Smith, Weaver, & Holland, 2014).
Handwriting Without Tears® (HWT), used in this study, is an established handwriting curriculum historically used by occupational therapy practitioners in traditional one-on-one service delivery but also designed for full-classroom implementation and instruction (Olsen & Knapton, 2008). The purpose of this study was to explore the effectiveness of HWT in general education kindergarten classrooms through teacher-led implementation supported by occupational therapist consultation.
Handwriting Without Tears Evidence
Recognizing and incorporating evidence-based interventions are important not only to occupational therapy practitioners but also to other school personnel. Therefore, it is important to review existing evidence for HWT. Most of the published studies support its use in a variety of contexts. Studies have been done on HWT implementation in general education through full-class instruction or with students who have identified disabilities through individual or small-group instruction.
At the preschool level, the full preschool HWT curriculum was supported through full-class implementation with at-risk students in inclusion classrooms at a rural Head Start (Donica, Goins, & Wagner, 2013; Lust & Donica, 2011). In addition, specific HWT techniques were supported for name writing and capital letter writing in full-class, small-group, and individual settings with preschool-age students (Carlson, McLaughlin, Derby, & Blecher, 2009; Griffith, McLaughlin, Donica, Neyman, & Robison, 2013; LeBrun, McLaughlin, Derby, & McKenzie, 2012). Studies have also supported the use of HWT in general education first-grade classrooms (Hape et al., 2014; Roberts, Derkach-Ferguson, Siever, & Rose, 2014; Salls, Benson, Hansen, Cole, & Pielielek, 2013). However, studies reviewing the use of HWT at the critical developmental grade of kindergarten are limited. Therefore, this study addresses this gap by asking the research question, Will students instructed using the kindergarten HWT curriculum at a private half-day kindergarten program have better end-of-year handwriting legibility scores than students in the same setting taught with teacher-developed lessons using the D’Nealian style of writing?
Method
Research Design
This pilot study used a static group comparison. Because the elementary school administrative decision to implement HWT in the kindergarten classrooms was made near the end of an academic year, a control group was identified as the current students who had been receiving teacher-led handwriting instruction during that year. Therefore, a traditional pretest–posttest design could not be used. Additionally, the instrument used was not standardized for children under age 6 yr, so standard scores could not be calculated for a pretest–posttest comparison because the children were not yet 6 yr old during the pretest. Likewise, because the school administration decided to implement the HWT curriculum schoolwide, a traditional control versus experimental classroom approach was not possible. Instead, the control group was identified as the group of students in kindergarten the year before HWT implementation. The research study was approved by the head of the school and the university institutional review board. Parent permission was received for all study participants.
Participants
The participants were half-day kindergarten students in a private school (kindergarten to 8th grade) in rural eastern North Carolina. Although the school did not specifically use the RtI model, the consultative approach of the occupational therapist with the teachers mirrors a commonly identified expanded role of occupational therapy practice with general education students. All students enrolled in kindergarten during the last month of school were invited to participate in the study as the control cohort. Control students subsequently learned handwriting using HWT in first grade the year after they participated in the study. Likewise, all students in the first-year experimental group (HWT 1) and the second-year experimental group (HWT 2) were invited to participate, but students were not included if they did not have parent permission, withdrew from the school or joined the school during the academic year, or were under age 6 yr at the time of data collection. Therefore, the sample sizes were n = 19 (out of 25) for the control group, n = 20 (out of 29) for HWT 1, and n = 20 (out of 39) for HWT 2. Student demographics are presented in Table 1.
Participant Demographic Characteristics (N = 59)
Note. HWT = Handwriting Without Tears; M = mean; SD = standard deviation.
Measures
Data collection occurred consecutively over 3 yr and included data from the control group and HWT 1 and 2. Data collection was completed using the Test of Handwriting Skills–Revised (THS–R; Milone, 2007), which was designed to assess a child’s neurosensory integration skill and is implemented to gather data on either manuscript (print) or cursive writing. For this study, the manuscript assessment was used. The test is standardized for children ages 6 yr 0 mo to 18 yr 11 mo and consists of 10 separate subtests. The activities in these subtests are described in Table 2. The THS–R was administered to one class at a time and took about an hour per class.
Comparison of THS–R Mean Scaled Scores Between Control and Experimental Groups
Note. For all results, p < .05 is significant. p values were calculated while controlling for age and gender. HWT = Handwriting Without Tears; LC = lowercase; M = mean; SD = standard deviation; THS–R = Test of Handwriting Skills–Revised; UC = uppercase.
The THS–R was selected to measure differences in handwriting skills because it is standardized and allows for a variety of scores to be used for analysis. The overall standard score, scaled subtest scores for each of the 10 subtests, and subsequent percentile scores were determined. Scaled scores have a mean (M) of 10 and a standard deviation (SD) of 3, whereas standard scores have an M of 100 and an SD of 15. The test–retest reliability was .82 for the total test score, with interrater reliability ranging from .75 to .90 based on the authors of the assessment (Milone, 2007). Unfortunately, no standardized handwriting assessments exist with standard scores for students under age 6 yr, which limits the ability to collect standard scores from the beginning of the kindergarten year. Because the use of standard scores is ideal for data analysis, students younger than age 6 yr (72 mo) at the time of data collection (end of the school year) were excluded from data analysis.
Intervention
Throughout the kindergarten year, the control group received teacher-developed instruction using the D’Nealian style of writing, and HWT 1 and 2 learned printing through the use of kindergarten HWT. At the end of the kindergarten year, the students completed the THS–R to determine the quality of their handwriting skills. The end-of-year scores for the control group were compared with the end-of-year scores for HWT 1 and 2 and with both experimental groups combined (HWT combined).
Intervention Description.
HWT 1 and 2 were instructed by their classroom teachers, who followed the lesson plans in the HWT Kindergarten Teacher’s Guide (Olsen & Knapton, 2008). The kindergarten HWT curriculum included the following materials: wood pieces for capital letters with mat; slate chalkboards (one classroom used the stamp-and-see screens with similar teaching techniques as the slates because of the teacher’s aversion to chalk); Roll a Dough set; Rock, Rap, Tap, and Learn CD; and Letters and Numbers for Me student workbook (Olsen, 2008).
The lesson plans (Olsen & Knapton, 2008) required approximately 15 min per day of teacher instructional time, which was typically adhered to throughout the 2 yr. Each lesson was taught to the full class and typically began with a gross motor activity coordinated with a handwriting-related song on the Rock, Rap, Tap, and Learn CD. Next, the teaching guidelines (Olsen & Knapton, 2008) were followed to implement a learning activity, which was either forming specific letters with multisensory manipulatives or writing in the Letters and Numbers for Me workbook (Olsen, 2008). In addition to the formal handwriting instruction time, an occasional review activity, often using the manipulatives, was used as an independent morning work activity. A classroom assistant helped with materials in all classrooms throughout the 2 yr.
As part of the consultative role for this intervention, a registered occupational therapist (the author) or two occupational therapy graduate students were present in the classrooms during the handwriting lesson one time per week. This presence allowed the occupational therapy personnel to answer questions about the implementation of the curriculum and to provide occasional assistance to struggling writers. Lessons were implemented similarly for both HWT 1 and 2. Consultation by an occupational therapy practitioner with the teachers did not occur with the control group.
Interveners.
Three kindergarten teachers were involved in the study. Two teachers with 17 yr and 4 yr of teaching experience at the beginning of HWT 1 intervention participated in all three cohorts. The same two teachers who taught HWT 1 also taught the control group. After the control year, these two teachers attended a full-day printing and cursive training workshop on the HWT curriculum, which is recommended but not required of the program. The teachers worked together to develop their lesson plans based on the HWT Kindergarten Teacher’s Guide (Olsen & Knapton, 2008).
After HWT 1 completed kindergarten, an additional kindergarten teacher with approximately 6 yr of experience was added. The new teacher attended the same HWT training as the other two teachers, but this training occurred after HWT 2's school year had started. She collaborated with the former teachers to create her lesson plans and understand the materials.
Intervention Fidelity.
To address fidelity to instruction, approximately one lesson per week of HWT 1 was observed by the author. The author consulted with the teachers and provided feedback on the teaching strategies, checked for proper use of handwriting activities and verbal cuing, and assisted as needed to address handwriting needs of specific students. This process helped establish consistency in instruction for both HWT 1 and 2. Weekly visits from the occupational therapist were unnecessary during the second year because the teachers indicated they understood how to implement the program, but the author did periodically check in with the teachers to answer questions if they arose.
Data Collection
The THS–R assessments were coded and scored semiblindly. Handwriting assessments for the three cohorts (control, HWT 1, and HWT 2) were scored by trained occupational therapy graduate students at different times, so the scorers were not blind to the cohort. However, this kindergarten study was part of a larger study that included first-grade THS–R assessments and two additional administrations of the THS–R throughout the year (approximately 4 mo apart) for the experimental groups. Therefore, even though the scorers were aware of which cohort assessments they were scoring, they were blind to the grade level and when during the academic year the assessment occurred. Scorers were trained by the author and by the DVD included in the THS–R assessment. They scored four sample handwriting assessments and discussed their differences in scoring for consistency before scoring the participants’ assessments, and they were randomly assigned assessments to score. However, interrater reliability was not established formally.
Data Analysis
The scaled scores, standard scores, and percentile scores of the THS–R were used for data analysis. Data analysis was completed using IBM SPSS Statistics (Version 22; IBM Corp., Armonk, NY). Descriptive statistics were calculated to determine the M scores and SD for each group on each of the subtests to identify specific skills and on the overall standard score for legibility. In addition, analysis of covariance (ANCOVA) was used to make statistical comparisons, controlling for age and gender because of documented differences in handwriting skills between boys and girls (Graham, Berninger, Weintraub, & Schafer, 1998). Using ANCOVA, we compared the control group with HWT 1, HWT 2, and HWT combined. Treatment effect (Cohen’s d) was calculated for each subtest comparison and for the overall score of the THS–R. This calculation serves as a frame of reference for the effect of the intervention on the outcomes and is valuable because of the small sample size. The effect size is considered small if 0.20 ≤ d ≤ 0.49, medium if 0.50 ≤ d ≤ 0.79, and large if d ≥ 0.80 (Cohen, 1992; Thalheimer & Cook, 2002).
Results
This study sought to determine whether students who completed kindergarten having learned handwriting skills from the HWT kindergarten printing curriculum would outperform students who learned printing from teacher-developed methods in D’Nealian-style writing in handwriting legibility skills. Table 2 includes the mean end-of-year THS–R scores for the control group compared with such scores for HWT 1, HWT 2, and HWT combined.
The experimental groups outscored the control group in all THS–R subtests and on overall score. ANCOVA showed that in 6 of the 10 subtests, both HWT 1 and HWT 2 scored significantly higher than the control group (p < .05). In addition, ANCOVA showed that in 7 of the 10 subtests, HWT combined scored significantly higher than the control group (p < .05). Figure 1 illustrates that the control group scored below the mean in 9 of the 10 subtests while almost all of the subtests for the experimental groups were above the mean (28 of 30). The control group performed at mean 36.63 percentile, whereas HWT 1 performed at mean 61.85 percentile and HWT 2 at mean 68.10 percentile. Refer to Table 2 for the specific results for scaled and standard scores.
Mean scaled scores on end-of-year Test of Handwriting Skills–Revised subtests.
Treatment effect (Cohen’s d) was calculated for each comparison. For all experimental groups (i.e., HWT 1, HWT 2, and HWT combined), a large treatment effect was found for 5–7 subtests and a medium treatment effect was found for 1–2 subtests. For HWT 2 and HWT combined, a small treatment effect was found for 2 subtests. In addition, for all experimental groups, a very large treatment effect was found for the overall standard score (d = 0.81 for HWT 1, 1.03 for HWT 2, and 1.00 for HWT combined). The treatment effects for each subtest and overall standard scores are included in Table 2.
Discussion
The results from this pilot study show that the students who received handwriting instruction with HWT outperformed the control group consistently and across all skill areas. For uppercase letters, students in the HWT groups demonstrated a large treatment effect for printing the uppercase alphabet from memory (airplane), uppercase from dictation (butterfly), and copying selected uppercase letters (tree). These results are not surprising because the HWT curriculum begins with the students learning all uppercase letters before lowercase ones. The curriculum is diligent in instructing each uppercase letter individually in a developmental sequence through multiple multisensory mediums and in the workbook Letters and Numbers for Me (Olsen, 2008).
Lowercase letters are taught in the HWT curriculum after all of the uppercase letters because of their complexity in line placement, stroke, and sequence. Each letter is instructed through multisensory techniques in a developmental sequence. The students in the HWT groups showed the largest treatment effects for printing lowercase from memory (bus). The experimental groups also demonstrated small to medium treatment effects for copying selected lowercase letters (horse) and medium to large treatment effects for copying words from a model (truck). Printing lowercase letters from dictation (frog) was not statistically significant but did demonstrate small treatment effects with the HWT 2 and HWT combined groups.
The formation of numbers from memory was also included in the THS–R (bicycle). Although the experimental groups demonstrated higher scores, differences were not statistically significant. The students began learning number formation early in the year through their math curriculum, which varied somewhat from the HWT number formations. Therefore, their introduction to this skill was not initially through the HWT curriculum.
The results for the skill level comparisons between individual upper- and lowercase letters were expected. However, large treatment effects were consistently seen for copying two sentences (book), and medium to large treatment effects were seen for writing words from dictation (lion). These effects were higher than expected because these skills are more complex than printing individual letters and are typically not well established at the end of kindergarten. Anecdotal teacher feedback supported these results, and teachers were pleased with the skills of the experimental groups.
Research supports the effectiveness of HWT in a full-class general education classroom (Hape et al., 2014; LeBrun et al., 2012; Roberts et al., 2014; Salls et al., 2013); however, there is a gap in the literature for HWT use in kindergarten classrooms. This study was done in an effort to help bridge that gap while demonstrating how occupational therapy practitioners may serve as consultants to teachers in general education as supported by the current legislation.
Limitations and Future Research
Although this study has produced some important results, it has limitations that must be considered when interpreting its usefulness in evidence-based practice. First, because of the way the school administration chose to implement the curriculum, it was impossible to do a pretest–posttest comparison for each of the groups. However, pretests were done on each of the HWT groups to ensure that their scores at the beginning of the treatment year were not higher than those of the control group at the end of kindergarten. In addition, a confounding variable is that the THS–R was completed 3 times by each experimental group but only one time by the control group. However, approximately 4 mo passed between administrations to minimize learning effects.
Limitations also exist because interrater reliability was not formally established, and although some level of blinding occurred, it was not complete across all three data collection points. Lack of formal intervention fidelity monitoring is somewhat of a limitation. However, this study was designed to see whether the curriculum instructed by a teacher with collaboration from occupational therapy was effective. Therefore, it was important to let the teachers implement the curriculum as they saw fit using the guidelines provided by the curriculum as a guide.
Further research should include more involvement of occupational therapy practitioners, not only with individual students but also at the classroom and system level. To establish more evidence regarding best practices in handwriting instruction, further research should be done at the kindergarten level. However, before that research, the development of a psychometrically sound tool to measure handwriting legibility skills for the kindergarten population should be considered.
Implications for Occupational Therapy Practice
This study supports collaborative efforts between teachers and occupational therapy practitioners in teaching handwriting skills and gives occupational therapy practitioners more evidence on which to base recommendations in school problem-based teams and curriculum committees. The results have the following implications for occupational therapy practice:
Occupational therapy practitioners must continue to advocate for their involvement in general education problem-solving teams at the school level, which may include providing recommendations for handwriting curriculum.
Occupational therapy practitioner consultation with teachers can be successful in implementing handwriting curricula.
HWT is an evidence-based curriculum that can be recommended by occupational therapy practitioners for effective printing instruction at the classroom or institutional level.
Footnotes
Acknowledgments
The author thanks the students and teachers who were involved in this study and the East Carolina University master of science in occupational therapy graduates who assisted with data collection: Simone Barnes, Kristen Gibbs, Anne Thomas, and Caitlin Zawistowicz. In addition, Suzanne Hudson, associate professor at East Carolina University, consulted on the statistical analysis. The author also thanks HWT, which provided discounted trainings for the teachers and graduate students involved in the study.