Abstract
Proficient handwriting is an essential component of literacy and an important foundation needed to support a child’s academic success. It is estimated that children in elementary school spend 31%–60% of each academic day occupied with fine motor tasks, with the majority of that time involving handwriting (McHale & Cermak, 1992). Research shows that 10%–30% of elementary school children struggle with handwriting (Karlsdottir & Stefansson, 2002). The education literature documents the consequences of poor handwriting for long-term academic performance and future employment (Berlin & Sum, 1988; Sum, Kirsch, & Yamamoto, 2004). Graham, Harris, and Fink (2000) suggested that children who experience difficulty mastering handwriting may avoid writing, potentially leading to a lack of participation in an important daily occupation for children and compromising the long-term outcomes of education.
Occupational therapy practitioners provide schools with universal strategies to support children’s acquisition of good handwriting habits (Case-Smith, Holland, Lane, & White, 2012; Schneck & Amundson, 2010). If a child’s handwriting does not improve through a teacher’s instructional practices, occupational therapy practitioners use a variety of assessments to evaluate the child’s visual–motor readiness and sensory and fine motor foundational skills needed for handwriting (van Hartingsveldt, De Groot, Aarts, & Nijhuis-Van Der Sanden, 2011). Practitioners often recommend specialized developmental handwriting programs, foundational skill building, and materials and environmental adaptations to improve handwriting (Bissell, Fisher, Owens, & Polcyn, 1998; Schneck & Amundson, 2010; Woodward & Swinth, 2002). Occupational therapy research has supported the effectiveness of handwriting interventions (Case-Smith, 2002; Case-Smith et al., 2012; Denton, Cope, & Moser, 2006; Feder & Majnemer, 2007; Howe, Roston, Sheu, & Hinojosa, 2013; Peterson & Nelson, 2003; Weintraub, Yinon, Hirsch, & Parush, 2009, Zwicker & Hadwin, 2009). Moreover, Berninger and Amtmann (2003) provided evidence that handwriting intervention can have a significant impact on other academic skills, such as spelling.
In recent years, a trend has developed to involve children in the assessment and intervention process, including self-evaluation and collaboration in setting educationally relevant occupational therapy goals (Missiuna, Pollack, & Law, 2004; Schneck & Amundson, 2010). Research indicates that self-evaluation plays a key role in fostering improved learning and higher degrees of motivation (Alderman, 1990; Toglia, 2011). When students reflect on their performance, it encourages them to set higher goals and commit more effort to achieve them (Rolheiser & Ross, n.d.; Ross, 2006). Moreover, researchers believe that children maximize their achievement in school when they have personal goals to attain, feel they have control over their successes and failures, and are motivated intrinsically to learn (Marchant, Paulson, & Rothlisberg, 2001).
Self-assessment promotes opportunities for students to identify their thoughts and feelings about their learning and performance at the cognitive, affective, and operative levels. When children are encouraged to critically analyze their work through reflection and self-assessment, they have the opportunity to take control of their learning. This process promotes self-regulation, independence, and a sense of ownership of the learning process, leading to the acquisition, maintenance, and generalization of skills (Graham, Harris, & Mason, 2005; Graham, Harris, & McKeown, 2013; Missiuna et al., 2004; Shapiro, 2011; Young, Yoshida, Williams, Bombardier, & Wright, 1995).
Further, research has indicated that goals established by others are not nearly as motivating as those established by people for themselves. Bandura and Schunk (1981) studied goal attainment and showed that child-centered goal setting was the most effective method of enhancing competencies, self-perceptions of efficacy, and intrinsic interest. This trend of involving the child is in keeping with the earlier writings of Meichenbaum (1977), which emphasized the importance of involving clients as partners in the therapy process (Polatajko & Mandich, 2004) and is consistent with client-centered practice (Fearing & Clark, 2000; Missiuna et al., 2004).
Children’s involvement in the identification of their needs and the selection of their intervention goals is likely to increase their motivation to learn, thereby promoting academic achievement (Graham et al., 2013; Schunk, 1996). Thus, tools that enable student self-assessment and goal setting may elicit more powerful and enduring levels of student engagement in the learning environment (Toglia, 2011; Zlotnik, Sachs, Rosenblum, Shpasser, & Josman, 2009). However, this key component often is missing in formal handwriting assessment and intervention.
This article presents two studies that examined the development and validity of Here’s How I Write: A Child’s Self-Assessment and Goal Setting Tool (HHIW; Goldstand, Gevir, Cermak, & Bissell, 2013), which was designed to provide occupational therapy practitioners and educators with an assessment that includes the child in the process of identifying his or her relative strengths and areas of weakness in handwriting and helps the child collaborate in setting goals.
Description of the HHIW
The HHIW was modeled after the original Hebrew-language child’s self-assessment of handwriting developed for Israeli Hebrew-speaking children, Kach Ani Kotev (Goldstand & Gevir, 2006, 2012). Kach Ani Kotev was the first pictorial self-assessment tool designed to examine schoolchildren’s perception of their own handwriting skills. The HHIW is a translated, adapted, and elaborated version of the Hebrew handwriting assessment and consists of a card game interview and goal setting process for English-speaking children. The English version includes child-driven data collection and progress monitoring as part of the process using samples from the child’s existing curriculum assignments.
The HHIW is a criterion-referenced handwriting assessment with standardized administration procedures. It consists of a picture card interview in which the examiner presents a child with 24 cards illustrating various aspects of handwriting, including feelings about handwriting, handwriting performance, and physical factors (Figure 1). Each side of the 24 cards has a positive and a negative attribute, and the examiner asks the child, “Which is more like you?” The examiner then asks the teacher about the same 24 aspects of the child’s handwriting using a checklist format (Figure 2).
Sample cards from Here’s How I Write: A Child’s Self-Assessment and Goal Setting Tool.
Items and score sheet for Here’s How I Write: A Child’s Self-Assessment and Goal Setting Tool.
Each of the 24 items on the HHIW is scored on a scale of 1 to 4. In response to the question, “Which is more like you [or the target child]?” participants choose the attribute that best describes the handwriting and then qualifies the response in terms of all of the time or some of the time. All of the time is scored 4 on a positive trait and 1 on a negative trait. Some of the time is scored 3 on a positive trait and 2 on a negative trait. Scores range from 24 (a negative perception of the child’s handwriting) to 96 (a positive perception of the child’s handwriting). In addition, the number of items on which the child and teacher agree on a positive score (3 or 4) or negative score (1 or 2) is also summed, and a percentage of agreement score is calculated. Although a numeric score is provided, no cutoff scores are suggested to determine whether a student has good or poor handwriting because this tool is a child’s self-perception of his or her writing, not an objective measure of handwriting quality.
The 24 test items are divided into three categories: (1) affect, or how the child feels about his or her handwriting (2 items); (2) performance, or how the child thinks he or she performs various components of handwriting (19 items); and (3) physical factors, or body posture, stabilization of the page, and fatigue when writing (3 items).
The HHIW adds a child-centered dimension to standardized fine motor, visual–motor, and handwriting performance assessments. The assessment process and results establish a beginning conversation and negotiation between the child and teacher or occupational therapy practitioner to identify handwriting needs and set handwriting goals meaningful to the child. The instrument includes child-directed goal setting, progress monitoring, and data collection procedures, with case examples to illustrate its application. Results provide a basis for working in partnership with the child to create a handwriting improvement plan with measurable goals.
Study 1: Needs Assessment and Content Analysis
Objectives
The purposes of the first study were (1) to examine the therapists’ and educators’ perceived need for a child’s self-assessment of handwriting and obtain consultation from occupational therapists and educators on test item development and (2) to conduct a content validity study of the proposed test items.
Method
We obtained approval for this study from the University of Southern California Health Science Campus institutional review board (IRB).
Participants.
A total of 42 occupational therapists and teachers participated in the needs assessment and content analysis study. In Phase 1, needs assessment, the team included 6 occupational therapists and 2 teachers working with children with handwriting problems in the public schools in California. Of the 6 occupational therapists, 4 had practiced for 10 yr, including more than 5 yr in the public school system. Two therapists had been practicing in the public school setting for more than 5 yr and currently were teaching courses in the Handwriting Without Tears curriculum (Olsen, 2012). The teachers had been teaching in elementary schools for more than 5 yr; 1 was a first-grade teacher, and the other was a teacher on special assignment in the Response to Intervention process working with kindergarten through third-grade children with needs in language arts (fluency, phonetics, decoding, and penmanship).
Participants in Phase 2, content validity, were a sample of convenience and included 34 experts in work with children with handwriting challenges. Of these, 21 were occupational therapists (9 were university faculty teaching pediatrics), and 13 were teachers. Eighty percent of the experts were from California; the other 20% were from different parts of the United States. The majority of the therapists had more than 10 yr experience working with school-age children and either a master’s or doctoral degree. The study examined the results of these experts’ opinions on the items on the HHIW.
Procedures.
We conducted a needs assessment and online survey to support the content validity of the HHIW. In the needs assessment, we interviewed the occupational therapists and educators about whether they thought that a child’s self-assessment of handwriting was important to measure, and they reviewed the test for clarity and comprehensiveness. We used the following questions as an interview guide:
What percentage of your [occupational therapists’] referrals relate in some way to handwriting problems?
What tools do you use in identifying handwriting needs and making recommendations?
At what age do you think a child is capable of self-assessing the status of his or her own handwriting?
Would a child’s self-assessment of handwriting be valuable to you in practice?
Do you anticipate any problems with the use of such a tool?
The occupational therapists and teachers also provided suggestions for revising the pictures and wording of the HHIW items.
For the content analysis, we created an online survey to obtain feedback from a larger number of experts. The following four questions were asked about each of the items: (1) Does the item measure handwriting? (2) Is the wording clear and understandable? (3) Do the pictures accurately illustrate the words? (4) Is the assessment comprehensive and free from cultural and gender bias?
Data Analysis.
Results of the needs assessment phase were analyzed descriptively. Results of the content validity phase were analyzed by percentage of occupational therapists or educators agreeing or disagreeing with each item. Feedback comments were collated and reviewed.
Results
Needs Assessment.
The occupational therapists and teachers reported that handwriting was a major concern among teachers and parents who referred children for occupational therapy. The following are summaries of the responses to each question:
Percentage of occupational therapy referrals for handwriting problems: The occupational therapists reported that handwriting was a nearly universal concern among teachers and parents who referred children to occupational therapy. The percentage of referrals with handwriting concerns these occupational therapists received ranged from 75%–90%.
Tools used in identifying handwriting needs and making recommendations: The method the occupational therapists and teachers used most frequently to evaluate students’ handwriting needs was a subjective evaluation of work samples followed by administration of the Beery–Buktenica Developmental Test of Visual–Motor Integration (Beery, Buktenica, & Beery, 2006), the Peabody Developmental Motor Scales (Folio & Fewell, 2000), the Bruininks–Oseretsky Test of Motor Proficiency (Bruininks, 1978), the Handwriting Without Tears Print Tool (Olsen & Knapton, 2006), or the Evaluation Tool of Children’s Handwriting (Amundson, 1995). Many of the occupational therapists used assessments that focused on foundational factors versus ecological factors, such as the child’s opinion, in the context of everyday handwriting.
Age of handwriting self-assessment capability: The youngest age at which the occupational therapists and teachers thought a child would be able to self-assess the quality of his or her handwriting was second to third grade, with the exception of one therapist and one teacher who thought that a first-grade student would be able to self-assess toward the end of the school year.
Value of a child’s self-assessment of handwriting in practice: Interviewees unanimously agreed that a child’s self-assessment of handwriting would be a valuable addition to the current method of evaluation if it were able to inform the instruction and intervention process. Comments included the following: “This tool may be useful in the prereferral process and eliminate the need for a complete assessment.” “It is a tool that potentially teachers as well as OT’s could use collaboratively.” “I like the idea of involving the student in the assessment process.”
Problems anticipated with the use of such a tool: The teachers and occupational therapists identified the following potential concerns: “A child with a handwriting problem related to visual perception may not perceive the difference in neat vs. poor handwriting.” “Some children with poor handwriting may not be capable of improving their own handwriting, even though they agree it needs improving.” “Sometimes the problem is not with the recognition of poor letter formation but in the mental translation of what to write and how to organize thoughts and words in an organized manner… . I have children who have neat writing in OT and poor handwriting in class.”
Reviewers also provided feedback on individual test items. On the basis of this feedback, we made efforts to eliminate unnecessary visual stimuli in the pictorial representations and to make each item as clear as possible.
Content Analysis: Online Survey.
When asked if they thought that a child’s self-assessment of handwriting was important to measure, the majority of the 34 experts (91%, n = 31) reported that it was either important or very important. In a review of the items and pictorial representations, 74% (n = 25) of the experts agreed or strongly agreed that the test was free of cultural bias, and 18% (n = 6) neither agreed nor disagreed. Both teachers and occupational therapists (94%, n = 32) agreed that the test was free of gender bias. The experts indicated that all areas of handwriting were covered by this assessment, but one teacher commented that the context (e.g., independent writing vs. writing in an adult-monitored situation) also should be considered.
For the content analysis, we asked the experts the following questions for each item: (1) Does the item measure handwriting? (2) Is the wording clear and understandable? (3) Do the pictures accurately illustrate the words? When examining the experts’ feedback, we paid particular attention to those items in which fewer than 80% agreed or strongly agreed or more than 10% disagreed or strongly disagreed. The experts agreed that 22 of 24 items (92%) on the HHIW measured handwriting. Some (44%, n = 15) stated that the sample item on the ability to solve puzzles was not a good measure of handwriting. However, we designed this item for use in explaining test directions and did not intend it to be a measure of handwriting; thus, it is not included in the scoring.
The experts also had a high level of disagreement (38%, n = 13) that the item “I do not make spelling mistakes” was a good measure of handwriting. Because spelling is often considered a measure of language and not handwriting, we eliminated this item from the handwriting assessment, although it remains in the Israeli version. Further item analysis showed that some of the items related to affect and others to performance and physical factors. The experts recommended that the items be reordered and grouped into three factors: affective, performance, and physical factors. We addressed this suggestion in the test revision.
The experts considered the wording of 22 of the 24 handwriting items (92%) to be understandable for children in second through fifth grade. They considered two items to be confusing: (1) the item regarding the letters that should be “tall,” such as b, d, f, h, and k; and (2) the item regarding letters that should go below the line, including g, j, p, q, and y. We clarified the wording of both of these items. In addition, the experts thought that the item “I write complete answers in tests and assignments” was confusing. The intent of this item was to gather information about the quantity of written work versus the content, and we changed the wording on this item.
The experts thought that in general, the pictures illustrated the words describing each handwriting component, with the exception of four of the 24 cards (17%). They thought that the picture regarding the “tall” letters was confusing, and we revised this picture to eliminate visual distractions. On several cards, the experts thought that the example of the child’s writing on the card was too small for children to see the difference between the good and poor handwriting; we revised these cards as well. Thus, in summary, one item (spelling) was eliminated, and several of the remaining items underwent word or pictorial revisions, reordering, and categorization.
Summary
The occupational therapists and teachers we surveyed unanimously agreed that a child’s self-assessment of handwriting would be a valuable addition to the assessments available in the fields of occupational therapy and education. Item analysis supported the content validity of the HHIW.
Study 2: Construct Validity
Objectives
The second study addressed the construct validity of the HHIW. It examined what children thought about their own handwriting in comparison with what their teachers thought and compared the responses of children with poor handwriting to those without handwriting problems to determine whether the tool discriminated between the two groups.
Method
We obtained approval for the study from the University of Southern California Health Science Campus IRB. Parents and teachers provided consent, and children provided assent.
Participants.
The participants were a representative sample of 40 students (13 girls, 27 boys) ages 7–11 yr in Grades 2–5 and their teachers in three public schools in one urban California school system. The teachers identified 20 children with handwriting problems (e.g., illegible, messy, or disorganized writing; mean age = 8.65 yr, SD = 0.93; 15 boys, 5 girls) and 20 children with good handwriting (mean age = 8.54 yr, SD = 0.79; 12 boys, 8 girls). The children were 60% Hispanic, 28% White, 5% Asian, and 7% of two or more races or ethnicities. Children who did not have sufficient language processing skills to understand the HHIW card items were excluded from the study.
Administration and Scoring.
An occupational therapist administered the HHIW to each child one on one in either the classroom or a quiet room nearby. The test took approximately 15 min to administer.
Data Analysis.
Scores of children with good and poor handwriting were compared using unpaired t tests to examine the total score and each of the category scores for affect, performance, and physical factors. The total child and teacher response scores were compared using a paired t test and percentage agreement.
Results
The results showed statistically significant differences between the children with good handwriting and those with poor handwriting. Children with poor handwriting rated their handwriting significantly lower than children with good handwriting in the overall score (Table 1) as well as in each of the three categories of affect, performance, and physical factors (Table 2). The correlation coefficient of all of the children relative to their teachers was r = .62, showing overall moderate correlation between how teachers and children responded to the items. Teachers and children with good handwriting agreed on the status of the child’s handwriting, and the mean differences were not statistically significant. However, although children with poor handwriting rated themselves lower than those with good handwriting, their teachers rated them more poorly than they rated themselves (Table 1).
Comparison of Mean Total Scores for Good and Poor Handwriting Groups
Note. M = mean; SD = standard deviation.
Comparison of Mean Scores for Affect, Performance, and Physical Factors for the Good and Poor Handwriting Groups
Note. M = mean; SD = standard deviation.
Discussion
It is not surprising to find that children who have poor handwriting are indeed aware of their writing ability and feel that their handwriting is not good. Significant differences in ratings were found between the children with good and poor handwriting, both in their self-assessment and in the assessment of their teachers. This finding indicates that children are able to self-assess their own handwriting performance. However, a discrepancy was found between what children with poor handwriting thought of their handwriting and what their teachers thought, with teachers rating the children lower than children rated themselves. Several interpretations of this finding are possible: The children may have been aware of their problems but unaware of the extent and severity of those problems; the children may have been reluctant to score themselves at the bottom of the rating scale; and the teachers were not blinded to which group the children were in, which might have created a bias such that the teachers rated those children more severely.
Limitations and Future Research
This study has several limitations. The majority of participants in the content validity study and all participants in the construct validity study were from California and thus do not represent the entire population of students, teachers, and occupational therapists in the United States. Additionally, because the teachers in the construct validity study themselves had selected and referred the students with poor and good handwriting, the teachers may have been biased when completing the checklist assessment of the child’s writing. A study in which teachers complete the HHIW assessment on a random selection of students not identified by their handwriting ability would be another way to compare teacher and student assessments. Finally, this article presents information on the self-assessment and not on the goal-setting portion of the tool; a series of case studies on use of the HHIW in goal setting will be presented in a separate article.
Future research is needed to examine the reliability and effectiveness of the HHIW in practice, including whether intervention using a child self-assessment and goal-setting approach results in improved handwriting for school-based written work. It is important to recognize that this tool does not assess children’s handwriting quality but rather the child’s self-perception of his or her writing and is best used when accompanied by a performance-based evaluation of the child’s writing.
Implications for Occupational Therapy Practice
In occupational therapy practice, inclusion of a child in assessment, goal setting, and progress monitoring is key to enhancing his or her motivation, performance, and self-efficacy (Graham et al., 2013; Missiuna et al., 2004; Polatajko & Mandich, 2004; Schneck & Amundson, 2010; Toglia, 2011; Zlotnik et al., 2009). The HHIW is a new tool that incorporates these components in handwriting evaluation. The findings described in this article have the following implications for occupational therapy practice (Goldstand et al., 2013):
This research shows that the HHIW discriminates between children with and without handwriting problems and provides a direct method to identify and address handwriting concerns in the context of a child’s curriculum expectations.
The assessment can be used with other handwriting assessments and can aid a child’s teachers and occupational therapy practitioners in understanding his or her handwriting needs and developing methods for change.
The HHIW assessment process develops a child’s self-awareness, self-evaluation, and goal setting ability. Children engaged in the assessment and goal setting process become active, engaged learners.
The test is an ecologically valid assessment that assesses a child’s handwriting in the context of classroom performance and actively engages both the child and teacher in setting and monitoring goals.
The test is administered in an engaging child-centered card game interview format reflecting the childhood occupations of play, handwriting, and schoolwork.
The HHIW is useful in the Response to Intervention and individualized education program literacy and learning process in both general education and special education.
Conclusion
These studies show that children in second to fifth grade are able to self-assess their own handwriting, supporting the validity of the HHIW. The children reported that they enjoyed the card game interview about their handwriting and were interested in participating in a handwriting improvement plan. One student reported that he was happy to have a handwriting test that did not involve writing, and another student stated that she liked talking about her handwriting, including why it mattered and how to make it better. This feedback supports the use of the HHIW as a valuable tool to use as a platform to work in partnership with children and their teachers to better understand how to improve performance in handwriting.
Footnotes
Acknowledgments
We acknowledge Sarina Goldstand and Debbie Gevir, developers of the Hebrew version of the HHIW, and Keith Cross, Milk Row Studio, for his expertise in educational product design and development. Many thanks to Mary Grace Assistant Superintendent, Anaheim City School District, for her support of data-driven services.