Abstract
Challenges with handwriting can have a negative impact on academic performance, and these challenges are commonly addressed by occupational therapy practitioners in school settings. This systematic review examined the efficacy of curriculum-based interventions to address children’s handwriting difficulties in the classroom (preschool to second grade). We reviewed and computed effect sizes for 13 studies (11 Level II, 2 Level III) identified through a comprehensive database search. The evidence shows that curriculum-based handwriting interventions resulted in small- to medium-sized improvements in legibility, a commonly reported challenge in this age group. The evidence for whether these interventions improved speed is mixed, and the evidence for whether they improved fluency is insufficient. No clear support was found for one handwriting program over another. These results suggest that curriculum-based interventions can lead to improvements in handwriting legibility, but Level I research is needed to validate the efficacy of these curricula.
Handwriting difficulties are observed in 10%–30% of school-age children with and without identified disabilities (Feder & Majnemer, 2007). Children experiencing handwriting impairments tend to have lower achievement in mathematics, lower verbal IQ, and greater attention difficulties than their peers without impairments (Sandler et al., 1992), resulting in decreased ability to interact and engage in classroom settings. Poor handwriting can also lead to limited compositional fluency (Graham, Berninger, Weintraub, & Schafer, 1998), issues with taking legible notes and reading them later, and more time needed to finish assignments (Graham, 1992). Moreover, handwriting impairments have been linked to reduced working memory capacity and lower reading and spelling scores (McCarney, Peters, Jackson, Thomas, & Kirby, 2013), suggesting that handwriting challenges early in life may have cascading negative effects on learning and academic performance.
To address handwriting difficulties, several curriculum-based handwriting programs have been developed. These programs are taught within the classroom setting and are geared toward improving handwriting in all children, not just those exhibiting difficulties. Occupational therapy practitioners in the schools often facilitate these interventions; handwriting deficiencies are one of the primary causes for referral to occupational therapy among school-age children (Barnes, Beck, Vogel, Grice, & Murphy, 2003).
Handwriting Without Tears (HWT; Olsen & Knapton, 2008), a developmentally and multisensory-based handwriting curriculum, can be implemented in the classroom by both teachers and occupational therapy practitioners. The Write Start program (Case-Smith, Holland, & Bishop, 2011; Case-Smith, Holland, Lane, & White, 2012), another cotaught classroom-embedded intervention, is aimed at promoting writing fluency in grade school children of all ability levels. These and other curriculum-based programs (see Table 1) target handwriting performance in children’s classroom setting. The curriculum-based approach aligns with the Every Student Succeeds Act of 2015 (Pub. L. 114-95), a reauthorization of the No Child Left Behind Act of 2001 (Pub. L. 107-110), which allows schools to address the needs of all students but particularly focuses on children who are not meeting academic standards. Aligning handwriting interventions with classroom curricula is thought to promote greater generalization of skills to handwriting-based activities within the classroom.
Handwriting Curriculum-Based Interventions Examined in the Reviewed Studies
Despite the availability of curriculum-based programs, little research has been conducted on the efficacy of these interventions in improving handwriting performance. A previous systematic review found that handwriting interventions (a blend of both curriculum-based and non–curriculum-based programs) were effective when they provided sufficient time for handwriting practice (Hoy, Egan, & Feder, 2011). However, this review did not specifically evaluate curriculum-based handwriting interventions, and the majority of the literature on curriculum-based programs has been published since the review.
Therefore, the objective of the current study was to systematically review the efficacy of curriculum-based handwriting programs in improving handwriting in classroom activities for children with and without identified disabilities. Combining our systematic review with effect size calculations from each study, we specifically aimed to examine (1) whether curriculum-based handwriting interventions in general made meaningful changes to children’s handwriting legibility, speed, and fluency; (2) whether specific curricula rendered the largest treatment effects; and (3) whether specific characteristics of curricula (e.g., age at intervention, length of intervention) led to more substantial treatment effects.
Method
Search Strategy
We conducted a systematic search of the literature to identify curriculum-based handwriting interventions for children. The previous systematic review of handwriting interventions covered December 1978 to January 2010 (Hoy et al., 2011). The current review included studies of curriculum-based handwriting interventions published from January 2006 to December 2015. Figure 1 shows the number of studies identified, screened, eligible for, and included in the systematic review. With the help of a medical librarian, our team systematically searched the following databases: PubMed, EBSCOhost (including Academic Search Premier), CINAHL Plus With Full Text, Education Full Text, ERIC, MEDLINE, PsycINFO, Social Sciences Full Text, SocINDEX with full text, and OTseeker. For PubMed, key terms included child* and handwrit* and (intervention or therapy or program). For EBSCOhost, terms included child* (and handwrit* intervention or handwrit* program). For OTseeker, the broad term “handwriting” was used to encompass a wide range of articles. The searches were further narrowed by the use of filters, including peer-reviewed journal articles, publication within the past 10 yr, and clinical trials.
Flow diagram of articles identified, screened, eligible for, and included in the systematic review.
The search terms were developed to capture relevant articles and to ensure that the terms relevant to the specific thesaurus of each database were included. Additionally, the American Journal of Occupational Therapy was hand searched to ensure that all appropriate articles were included.
Selection Criteria
Articles selected for review included those that had used handwriting interventions and curriculum-based programs for children in preschool through fifth grade. We chose to exclude articles addressing children above the fifth-grade level to focus on the years when children typically learn handwriting fundamentals. We included studies of curriculum-based handwriting programs used for children both with and without identified disabilities, who together form the target population of these interventions. Other inclusion criteria were interventions that took place in a general education classroom, interventions longer than one session, and interventions with a clear beginning and end. Specific exclusion criteria were studies with adult participants, interventions implemented outside the classroom setting, and studies that lacked a distinguishable intervention. The studies used in our review were assessed for outcomes related to overall handwriting performance, such as legibility, writing speed, and fluency.
Effect Size Computations
Using the reported means and standard deviations published in each study, we calculated Hedge’s g using the compute.es package (Del Re, 2013) in R (R Core Team, 2015). Hedge’s g is an effect size measure that permits comparison of the size of the intervention effect across studies and measures. A Hedge’s g of 0.20 is considered a small effect, 0.50 is considered a medium effect, and 0.80 or greater is considered a large effect. Compared with Cohen’s d, Hedge’s g may provide a better estimate of effect size in small samples (Grissom & Kim, 2005). In the case of repeated measures analyses, we followed the recommendations of Morris (2008) by calculating Hedge’s g for the pre–post change in each group and then subtracting the Hedge’s g for the control group from the Hedge’s g for the treatment group. Because this procedure did not account for repeated measures, it may have led to decreased estimates of effect sizes for these analyses. Positive effect sizes represent the size of effect in the expected direction (i.e., faster, more fluent, or more legible handwriting), whereas negative effect sizes represent the size of the effect in the unexpected direction (i.e., slower, less fluent, and less legible handwriting).
Results
From the original search, we identified 252 studies matching our search terms. Of these studies, 99 were excluded because the titles did not include handwriting or children in preschool through fifth grade. Abstracts of the remaining 153 were screened, and 121 were excluded because of the absence of an explicit handwriting intervention or program (see Figure 1).
Supplemental Table 1 (available online at http://otjournal.net; navigate to this article, and click on “Supplemental”) summarizes the 13 included studies. Levels of evidence were assigned on the basis of AOTA guidelines (Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996). There were 0 Level I studies, 10 Level II studies, 2 Level III studies, and 1 study we classified as Level II–III because it had two distinct intervention groups but no control. Several curricula were examined by the 13 studies, including Write Start (Case-Smith, Holland, & Bishop, 2011), HWT (Olsen, 2003; Olsen & Knapton, 2008), HWT–Get Set for School (Olsen & Knapton, 2008), Peterson Directed Handwriting Curriculum (Nelson, 2006), the Fine Motor and Early Writing Pre-K curriculum, the Size Matters Handwriting Program (SMHP; Moskowitz, 2009), Write Direction (Taras, Brennan, Gilbert, & Eck Reed, 2011), and Handwriting Clubs (Howe, Roston, Sheu, & Hinojosa, 2013). Table 1 presents a brief description of each curriculum.
Risk of Bias
Because the studies under review were nonrandomized, we assessed risk of bias according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Moher et al., 2009) guidelines using ROBINS–I (Sterne et al., 2016). As seen in Supplemental Table 2 (available online), the majority of studies had low risk of bias across domains.
Effects of Level II Versus Level III Studies
Because Level III studies, by definition, do not have a control group, pre–post treatment effect sizes may be overestimated because they likely reflect not only improvements from the intervention but also maturation effects and benefits from completing the same or a similar measure twice. To examine whether the difference in effect sizes between the 10 Level II studies and the 3 Level III studies (including the Level II–III study) was robust, we contrasted the average effect sizes of the analyses. Level II studies had an average effect size of 0.32 (small to medium effect; range = −0.90 to 1.96), whereas Level III studies had an average effect size of 2.69 (very large effect; range = −0.19 to 9.98). The Level III effect sizes were on average eight times larger than the Level II effect sizes, suggesting that maturation effects play a large role in curriculum-based handwriting intervention research. Because of this large discrepancy, we report findings from all studies but effect sizes of only the Level II studies.
Effects of Curriculum-Based Handwriting Interventions
Handwriting Legibility.
Handwriting legibility was measured as an outcome in 12 of the 13 studies. Quality was considered a proxy for handwriting legibility. Because letter formation is a main component of legibility (Hammerschmidt & Sudsawad, 2004), letter formation was interpreted as legibility. Eight studies showed significant improvements in at least one component of legibility. Therefore, moderate evidence exists for improved handwriting legibility after curriculum-based handwriting programs. The reviewed interventions had an average effect size of 0.39 (range = 0.02 to 1.05), suggesting small to medium effects on legibility.
Handwriting Speed.
Nine of the 13 studies assessed handwriting speed or rate. Five of these studies found significant improvements in speed. However, 3 studies found no difference in speed, and Pfeiffer, Rai, Murray, and Brusilovskiy (2015) found that the intervention group became significantly slower after training than the control group. Therefore, the evidence is mixed regarding whether curriculum-based interventions enhance handwriting speed. The reviewed interventions had an average (mean) effect size on speed of 0.13 (range = −0.90 to 0.77). Because of the large range, we also calculated the median (0.22). Therefore, the intervention effects on handwriting speed were variable and, when averaged, were small to very small.
Handwriting Fluency.
Only 4 of the 13 studies assessed fluency, and all 4 investigated the Write Start program. Only 3 of these studies found significant differences in fluency at posttest. Because of the similarity of the studies and some inconsistencies in posttests, the evidence is insufficient for curriculum-based handwriting programs to improve fluency. Given the discrepancy in effect sizes of the 2 Level II studies that assessed fluency (range = −0.08 to 0.74), we did not calculate an average effect size.
Effects of Specific Curricula
We compared effect sizes for legibility and speed across the different curricula (see Figure 2). For legibility, an explicit handwriting program (Kaiser, Albaret, & Doudin, 2011) had the largest effect size, but this study was the only one to use this intervention. SMHP (Pfeiffer et al., 2015) and Write Start (Case-Smith, Holland, & White, 2014; Case-Smith, Weaver, & Holland, 2014) on average had medium to large effects on legibility. The Fine Motor and Early Writing Pre-K Curriculum (Donica, Goins, & Wagner, 2013), HWT (Donica, 2015; Donica et al., 2013; Lust & Donica, 2011; Roberts, Derkach-Ferguson, Siever, & Rose, 2014; Salls, Benson, Hansen, Cole, & Pielielek, 2013), Write Direction (Taras, Brennan, Gilbert, & Eck Reed, 2011), and intensive handwriting practice (Howe, Roston, Sheu, & Hinojosa, 2013) all had small or very small effects on legibility. However, many of these studies used active and rigorous control conditions, which might have diminished the size of these effects.
Effect sizes (Hedge’s g) for legibility and speed, by curriculum.
For speed, the explicit handwriting program (Kaiser et al., 2011) had the largest effect size. Write Start (Case-Smith et al., 2011, 2012; Case-Smith, Holland, & White, 2014; Case-Smith, Weaver, & Holland, 2014) and intensive handwriting practice (Howe et al., 2013) had small to medium effect sizes. Intriguingly, SMHP (Pfeiffer et al., 2015) had a small to medium effect size but in the opposite direction, suggesting that this curriculum significantly enhanced legibility while promoting slower writing.
Effects of Specific Characteristics of Curricula
Age at Instruction.
Effect sizes for legibility (17 effect sizes) and speed (8 effect sizes) were examined as a function of the grade at which the intervention took place (interventions that took place in Grades 1 and 2 were coded as 1.5). Legibility effects showed a medium-sized but not significant correlation with age at instruction, r = .33, p = .25. Speed effects did not vary according to age at instruction, r = −.06, p = .89.
Instruction Length.
The length of intervention varied across studies; however, all interventions lasted a minimum of 6 wk. The majority of handwriting programs lasted ≥12 wk and yielded handwriting improvements in at least one of the specified outcome areas. Given the variability in session time and frequency across interventions, we calculated an estimate of the total number of hours of intervention for each of the studies. Total hours of intervention ranged from 6 hr (Donica et al., 2013) to 90 hr (Donica, 2015), with the latter occurring over a 2-yr time span. Omitting the 90-hr intervention outlier, total intervention hours were not correlated with the effect sizes for legibility, r = .27, p = .37, or speed, r = −.11, p = .79.
Discussion
This systematic review aimed to examine the evidence for curriculum-based handwriting interventions to improve handwriting legibility, speed, and fluency. From our extensive literature search, 13 curriculum-based handwriting studies met inclusion criteria for review (10 Level II studies and 3 Level III studies). Conspicuously, there were no randomized controlled trials (Level I evidence). Our systematic review rendered two major findings: (1) Curriculum-based handwriting interventions in general demonstrated small to medium effects in improving legibility, and (2) although certain programs may be better suited for targeting speed versus legibility, other characteristics of the programs (i.e., age at intervention and hours of intervention) did not appear to influence efficacy.
Efficacy of Curriculum-Based Handwriting Interventions
The findings suggest that curriculum-based handwriting interventions can successfully elicit small- to medium-sized improvements in legibility. Although the size of these effects was not large, even small gains in legibility may be important because poor handwriting legibility can greatly compromise a child’s functioning in school and lead to lower grades (Graham, Harris, & Fink, 2000; Schneck & Amundson, 2010). These effect sizes may have been smaller than expected because many of the reviewed studies implemented active control groups (handwriting was taught, but in a different way) rather than passive control groups. Using an active control group is more rigorous but may result in underestimation of the size of the intervention effect.
In contrast, curriculum-based interventions did not appear to enhance handwriting speed. Speed effect sizes varied greatly, and the average speed effect size was small. One possible explanation is that when legibility and form are emphasized in a curriculum, slower handwriting production may result. Indeed, several studies demonstrated that when legibility improved, speed declined or showed no improvement (Howe et al., 2013; Roberts, Siever, & Mair, 2010; Weintraub & Graham, 1998). Another possible explanation is that improvements in letter quality may be observed before improvements in speed because of the additional practice time needed for speed to develop (Hoy et al., 2011). Consequently, speed effects may not be as evident in studies that focus on young learners.
Writing fluency was a variable of interest because the end goal of efficient handwriting is to allow children to focus on higher order aspects of writing. However, not enough studies measured fluency to be able to draw conclusions. This is a critical gap in the literature and a key avenue for future research because writing fluency likely reflects the more functional aspects of handwriting ability.
Program Characteristics That Demonstrated the Highest Efficacy
We calculated effect sizes for all studies in the systematic review to supplement our interpretation of the literature. This allowed us not only to estimate effect sizes across the whole body of evidence but also to compare effects across different curricula, ages, and lengths of intervention.
An important question has been whether one type of curriculum-based handwriting intervention outperforms the others. In other words, does it matter which curriculum a school uses? From our comparison of effect sizes, no one handwriting program appeared to outperform the other programs across all domains. Intriguingly, the Write Start program and the explicit handwriting program from Kaiser et al. (2011) were the only programs to have non–small effects on both legibility and speed. However, other programs had medium to large effect sizes in each of those domains (just not consistently across domains). Therefore, different programs may excel at targeting different outcomes.
In an ideal situation, the needs of the children in the classroom would dictate which curriculum is used. For example, our results suggest that SMHP may be best for classrooms for which the primary goal is legibility but not speed. Alternatively, for classrooms for which the primary goal is handwriting speed, the explicit handwriting program from Kaiser et al. (2011), Write Start, or the intensive handwriting program from Howe et al. (2013) might be best suited.
We also used the effect sizes to examine the ideal length of intervention, and we found that more intervention hours did not appear to lead to substantially larger handwriting improvements. This finding suggests that 6 wk of intervention may be sufficient, even though a previous review of curriculum-based and non-curriculum-based handwriting interventions suggested that handwriting interventions should occur at least two times per week for a minimum of 20 sessions to be effective (Hoy et al., 2011).
Interestingly, we found that the grade at which the intervention occurred had a nonsignificant but medium to large relation to how big the intervention-based legibility effects were, which suggests a trend for older grades to be associated with larger effects. Although this association was not statistically significant and should be interpreted with extreme caution, the size of the effects for different ages and grades might be useful in designing future research.
Limitations
A limitation of this review is that no Level I studies met inclusion criteria, restricting our ability to draw firm conclusions on the efficacy of curriculum-based handwriting interventions. The lack of Level I studies may be attributable to the fact that curriculum-based interventions, by definition, take place in the classroom, preventing random assignment of students to one condition or another. However, a large-scale study that randomly assigns different classrooms to the intervention or control condition would provide higher levels of evidence in support of curriculum-based interventions.
Another limitation is that our calculation of effect sizes did not account for repeated measures, which may have led to decreased estimates of effect sizes. We chose this as a conservative approach, but some of the effects may be underestimated. Other limitations include inconsistency in definitions of handwriting components (e.g., legibility), limited descriptions of participants, and lack of long-term follow-up in the studies reviewed.
Implications for Occupational Therapy Practice
The results of this study have the following implications for occupational therapy practice:
Curriculum-based handwriting programs, in general, appear to successfully target legibility in preschool, kindergarten, and young school-age children.
Specific curriculum-based handwriting programs may be better at targeting speed than legibility (or vice versa) and ideally should be selected on the basis of whether the primary need of the classroom is handwriting speed or legibility.
For the majority of children, 6 wk (∼15 hr) was enough to make gains in legibility. However, children with handwriting challenges may need more time. Future research is needed to determine the ideal length of curriculum-based handwriting programs.
A key need exists for future Level I research to examine curriculum-based handwriting.
Supplemental Material
Supplementary material for Curriculum-Based Handwriting Programs: A Systematic Review With Effect Sizes
Supplementary material, sj-pdf-1-aot-10.5014_ajot.2018.027110.pdf for Curriculum-Based Handwriting Programs: A Systematic Review With Effect Sizes by Courtney Engel, Kristin Lillie, Sarah Zurawski and Brittany G. Travers in The American Journal of Occupational Therapy
Footnotes
Acknowledgments
This work was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (Grant U54 HD090256 to the Waisman Center), the Brain and Behavior Research Foundation (NARSAD Young Investigator Award to B.G.T.), and the Hartwell Foundation (to B.G.T.). We thank Karla Ausderau and Lauren Bishop-Fitzpatrick for their insights and help with this systematic review. We also thank Michael Venner for his expertise and assistance with our database search.
*
Indicates articles included in the systematic review.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
