Abstract
Trainees could benefit from practicing orthotic fabrication on simulated hands with joint deformities. As a first step toward such training, we explored the use of a nonpathological model hand. Twenty-one participants were randomized into one of two groups that practiced using a person’s right hand or a model right hand. One week later, all participants returned for a transfer test in which they made one orthosis on a person’s left hand. All participants’ performance and orthoses were evaluated using a validated checklist and a global rating scale (GRS). Fabrication time for each orthosis also was recorded. The GRS score and fabrication time changed significantly over the course of practice. Trainees who practiced with the model hand made better orthoses during practice and on the transfer test, as measured with the checklist’s final product subscore. Instructional and contextual factors that may affect trainees’ performance and learning are discussed.
Keywords
Simulation has a long history of being used for education in various fields, such as aviation (Garrison, 1985) and the military (McKenzie et al., 2008). More recently, simulation training has gained acceptance within the health professions as a useful method for preparing trainees for clinical practice. The primary benefits of simulation include avoidance of the risk inherent in having junior trainees learn new skills on patients (Ziv, Small, & Wolpe, 2000) and reduction in teaching costs (Babineau et al., 2004).
In surgical training, the current paradigm for teaching technical skills is to establish preclinical proficiency before performing on patients. At the heart of this model are simulation tools such as virtual reality and bench-top devices, which allow for deliberate and risk-free practice (Aggarwal & Darzi, 2006; Reznick & MacRae, 2006) in which learners can commit and observe the consequences of errors (Issenberg, McGaghie, Petrusa, Gordon, & Scalese, 2005). Nursing and dentistry also use a variety of simulation modalities, such as artificial patients (Alinier, Hunt, Gordon, & Harwood, 2006; Terman, 2007), bench-top models, and virtual reality systems, to hone fine motor skills (Rees et al., 2007; Welk et al., 2008). Simulations also include role-playing with actors or standardized patients to improve physical examination and interpersonal and communication skills (Levine & Swartz, 2008; Oswald, Bell, Wiseman, & Snell, 2011). Research findings show that these training tools can have positive effects on learning (e.g., Van Sickle, Ritter, & Smith, 2006) and that skills learned in a simulated environment may transfer to the clinical environment (Anastakis et al., 1999; Hamilton et al., 2002). In fact, current research has moved beyond justifying the use of simulation; research is now focused on identifying best practices for simulation-augmented teaching (Issenberg et al., 2005; McGaghie, Issenberg, Petrusa, & Scalese, 2010).
Whereas the medicine, nursing, and dentistry communities all display robust research programs for simulation-augmented education, little is known about the extent of such activities in the rehabilitation professions (Yeung, Dubrowski, & Carnahan, 2013). A recent scoping review by Yeung et al. (2013) suggested that in the rehabilitation professions, simulation is being used for formative and summative evaluations and to enable the development of technical skills (primarily in physical therapy), clinical reasoning, and decision making. The simulation modalities used include procedural simulation with part-task trainers that simulate only selected elements of a patient or system (Graham, Clausen, & Bolton, 2010), computer-based systems (Stewart, 2001), and standardized or simulated patients (Hale, Lewis, Eckert, Wilson, & Smith, 2006; Zraick, Allen, & Johnson, 2003). In contrast to the field of surgery, Yeung et al. (2013) identified only three studies that explored how simulation-based educational techniques were used to optimally teach, improve, or reinforce technical skills in the rehabilitation professions. They also found little documentation of the long-term impact of simulation-augmented interventions on skill development or maintenance (Yeung et al., 2013).
Although motor skills are less central to the identity of the rehabilitation professions (unlike surgery or dentistry), practitioners are often required to perform technical clinical skills such as palpation, percussion, joint mobilization, and fabrication of orthoses. Orthotic fabrication, a skill unique to the rehabilitation professions, is a complex task often requiring the clinician to exercise motor and cognitive skills simultaneously. At the University of Toronto Department of Occupational Science and Occupational Therapy, students learn about orthotic principles by attending didactic sessions with experts, watching online videos that demonstrate the process, and receiving hands-on experience in a laboratory setting by fabricating orthoses on one another.
Even though practicing on another person’s hand is technically a form of simulation, it is limited: Students are learning to make orthoses only on young, healthy, typically functioning hands that would not require an orthotic device. Often, hands encountered in a clinical setting, such as those with rheumatoid arthritis, are sensitive to pain and have mild to severe joint deformities (Bielefeld & Neumann, 2005). For newly graduated occupational therapists or those with limited experience in a hand clinic, these clinical characteristics present a unique challenge and require the clinician to quickly adopt a new skill set under the time and performance pressures of clinical practice. This situation also puts the first few clients encountered by a trainee or recent graduate at higher risk of discomfort and suboptimal outcomes while the clinician acquires proficiency.
In the field of motor learning, the principles of practice specificity suggest that educators should intentionally design practice conditions that will prepare learners for the conditions under which the skills will be performed (Schmidt & Lee, 2011). Educators can achieve these conditions through sensory and motor specificity, context specificity, and transfer-appropriate processing. As such, the primary aim of simulation is to bridge the gap between what trainees learn in the classrooms and laboratories and what they will encounter with real clients or as clinicians. By giving trainees a physical model that closely resembles the physical characteristics and movement patterns of a hand with pathological characteristics or joint deformities, we believe that trainees will be better prepared for clinical practice upon graduation, which will ultimately benefit clients.
Before educators can evaluate the learning potential of working with artificial hands that demonstrate pathological characteristics, the learning potential of working with a nonpathological artificial hand needs to be established. Thus, the first step in the process of developing a simulation-augmented training program for orthotic fabrication is a proof of concept exercise to demonstrate that the simulation has some value in training rehabilitation practitioners. As such, our primary objective was to explore the utility of a nonpathological artificial or model hand as a teaching tool for orthotic fabrication by determining whether the type of hand used during practice (model or real) affects trainees’ practice performance or learning, as measured by a delayed transfer test.
Method
Research Design
We conducted an experimental, two-group study with a 2-hr learning intervention and 1-wk delayed transfer test. Using a random sequence generator, participants were randomized to one of two groups: the control group (Real Hand), which practiced making metacarpophalangeal (MCP) joint-stabilizing orthoses on a person’s right hand, or the experimental group (Model Hand), which practiced making MCP joint-stabilizing orthoses on a plastic right hand. This study received ethics approval from the University of Toronto Health Sciences Research Ethics Board, and all participants provided voluntary informed consent.
Participants
Participants were recruited from the University of Toronto Year 1 occupational therapy and physical therapy classes. Eligible participants were those who had no prior knowledge of or experience with making orthoses.
Task Description
The participants’ task began by taking a sheet of clear plastic, wrapping it around the hand (real or model), and then using a permanent marker to draw a pattern of the orthosis that they were going to produce. This pattern was then cut out and traced, using a grease pencil, onto a piece of low-temperature thermoplastic. The thermoplastic was submerged in water set to 95°C for approximately 15–20 s to make it easier to cut. Once the thermoplastic was cut out, it was submerged again for approximately 25–30 s so that it was soft enough to be properly molded. The thermoplastic was then removed from the heating pan, the excess water was dried off, and the thermoplastic was applied to the hand. Once it cooled and hardened, the participant was free to take it off the hand and make any cuts or modifications. This process continued until the participant determined that the orthosis was complete and properly fitted. Finally, the participant smoothed the edges and removed all pencil markings on the orthosis.
Apparatus and Materials
The plastic model hand was constructed from data obtained by a laser scan of a right male hand. Its outer shell was a thin, lightweight plastic, which was filled with standard caulking to provide extra stability to the model and protect it from breaks (Figure 1; Table 1). The primary investigator (Hagemann) provided his hand for the Real Hand group during practice and for both groups during the transfer test.
Photographs of the (A) palmar and (B) dorsal aspects of the model hand.
Comparison of Anatomical Measurements of the Model Hand and the Real Hand Used for Practicing Orthotic Fabrication
Note. MCP = metacarpophalangeal.
Sheets of solid, electric blue, 2.4 mm thick Aquaplast® Watercolors™ thermoplastic (Patterson Medical, Bolingbroke, IL), cut into 15 cm × 20 cm pieces, were used. The training video, created by one of the coauthors (McKee) as part of the curriculum at the authors’ institution, was a 9-min demonstration of an expert occupational therapist constructing the custom-fitted portion of the MCP joint-stabilizing orthosis for a healthy left hand. The video was augmented with verbal instructions as the skill was being performed. Participants viewed the video on a laptop computer that was set approximately 0.5 m in front of them. Other tools used to construct the orthoses included a pair of large scissors, pair of small scissors, spatula, permanent marker, grease pencil, and heating pan. Clear plastic bags were cut into 15 cm × 20 cm sections for the pattern-making portion of the task. Practice and transfer trials were recorded by a video camera 1.5 m away from the participant.
Procedure
Participants watched the training video once without interruption and received specific instructions about handling the thermoplastics. Practice then began and continued until either five orthoses were complete or 2 hr had elapsed. We also told participants that no expert feedback would be given and that they would be responsible for determining when each orthosis was complete. Participants were allowed unrestricted access to the training video throughout the practice session and could make as many modifications to the orthoses as they deemed necessary. Once they began a new orthosis, however, they were prohibited from making modifications or referring to any previously completed orthoses.
Participants returned 1 wk after the practice session to complete a delayed transfer test, in which they constructed one orthosis on the researcher’s left hand. Practice, verbal instruction, and reviewing of the training video were prohibited. The setup for the delayed transfer test was similar to that used for the practice sessions except that the laptop was not present.
Dependent Measures
Checklist and Global Rating Scale.
Checklists and global rating scales (GRSs) are commonly used as complementary elements for rating the performance of surgical trainees (Martin et al., 1997). Checklists are purely objective assessments of critical steps and the final product, whereas a GRS is a subjective rating of overall performance. Stefanovich, Williams, McKee, Hagemann, and Carnahan (2012) reported on initial validation tests of a checklist and GRS for fabrication of an MCP joint-stabilizing orthosis. Although psychometric properties have not yet been established, validation tests showed that both tools distinguished between novice and expert performers and that there were no significant differences between how performers were rated by a licensed occupational therapist rater and an informed nonclinician rater. The checklist consisted of 5 process-related and 10 product-related essential components of orthotic fabrication for a total score out of 15. The GRS consisted of 7 holistic performance items (each rated on a 5-point Likert scale) for a total score out of 35.
An informed nonclinician (Hagemann), trained by a hand therapist and experienced with orthotic fabrication, watched video recordings of practice trials and transfer tests and used the checklist and GRS to assess orthotic fabrication performance and quality of the final product. Blinded evaluation was not possible for the practice trials (group assignment was visible from the video), but the rater was unaware of group assignment for the transfer tests and unaware of the participant’s identity in each video.
Fabrication Time.
Time to complete each orthosis was obtained from the videotape and measured in seconds. Time began when the participant made the thumbhole cut in the plastic and ended when the participant indicated that the orthosis was complete. Time spent watching the video during a practice trial was subtracted from the fabrication time for that trial.
Data Analyses
Descriptive statistics were computed to describe the sample of participants. For each dependent measure described above, practice trials were analyzed using two-way mixed analyses of variance (ANOVAs) with practice group (Real Hand, Model Hand) as a between-subject factor and trial (1–5) as a repeated-measures factor. Separate independent t tests were used to compare the performance of groups at transfer for each dependent measure. We used SPSS Statistics Version 20 (IBM, Armonk, NY) for all analyses. Effects were considered statistically significant at p < .05, and Pearson’s correlation coefficient r effect sizes were calculated for between-group effects and considered small, medium, and large at .10, .30, and .50, respectively (Cohen, 1988, 1992, as cited in Field, 2009, p. 57).
Results
A sample of 21 first-year occupational therapy and physical therapy students participated in this study. The Real Hand group (average age = 24.3 yr) comprised 10 female participants, all of whom self-reported right-handedness. The Model Hand group (average age = 24.2 yr) comprised 2 men and 9 women; by self-report, 9 participants were right-handed and 2 were left-handed (1 male, 1 female).
Practice Performance
Analysis of checklist scores indicated no interaction between trial and group, F(4, 76) = 1.25, p = .299, and no main effect of trial, F(4, 76) = 2.27, p = .069 (Figure 2). However, we found a main effect of group, F(1, 19) = 4.52, p = .047, r = .44. For the GRS scores, Mauchly’s test (Field, 2009) indicated that the assumption of sphericity had been violated, χ2(9) = 17.28, p = .045; therefore, we used Huynh–Feldt estimates of sphericity to correct degrees of freedom (Field, 2009). The corrected results indicated a significant interaction between trial and group, F(3.5, 67.2) = 2.82, p = .037, and a main effect for trial, F(3.5, 67.2) = 13.81, p < .001 (Figure 2). Post hoc pairwise comparisons with Bonferroni corrections for multiple comparisons indicated that Trial 1 was significantly different from Trial 3 (p = .010), Trial 4 (p = .001), and Trial 5 (p = .002); also, Trial 2 was significantly different from Trial 3 (p = .017), Trial 4 (p = .004), and Trial 5 (p = .003). However, there was no main effect for group, F(1, 19) = 2.16, p = .158, r = .32.
Means and standard errors of the mean for each dependent measure by group (Real Hand, Model Hand): (A) checklist score, (B) global rating scale score, and (C) fabrication time.
Analysis of fabrication time revealed no interaction between group and trial, F(4, 76) = 1.15, p = .342, and no main effect of group, F(1, 19) = 2.02, p = .172, r = .31, but a main effect of trial, F(4, 76) = 4.83, p = .002 (see Figure 2). Post hoc pairwise comparisons with Bonferroni corrections indicated that Trial 1 was significantly different from Trials 3 (p = .014) and 4 (p = .014).
Because the checklist was the only measure to indicate that practice performance differed between the groups, we pursued post hoc analyses to elucidate potential explanations for this observation. We parsed the total checklist scores into product and process subscores to represent performance on the product- and process-related measures, respectively (Figure 3). For each subscore, we conducted two-way mixed ANOVAs with practice group (Real Hand, Model Hand) as a between-group factor and trial (1–5) as a repeated-measures factor. For the process checklist scores, Mauchly’s test indicated that the assumption of sphericity had been violated, χ2(9) = 17.96, p = .036; therefore, degrees of freedom were corrected using Huynh–Feldt estimates of sphericity. The corrected results indicated a significant interaction between trial and group, F(3.9, 73.8) = 3.66, p = .010, and no effect of trial, F(3.9, 73.8) = 2.26, p = .073, or group, F(1, 19) = 0.14, p = .718 (see Figure 3). For the product checklist scores, we found no interaction between trial and group, F(4, 76) = 0.81, p = .521, but we did find main effects of both trial, F(4, 76) = 2.65, p = .040, and group, F(1, 19) = 6.01, p = .024, r = .49.
Means and standard errors of the mean for the checklist (A) process and (B) product subscores by group (Real Hand, Model Hand).
Delayed Transfer Performance
Analysis of checklist scores indicated that the Model Hand group received significantly higher scores than the Real Hand group, t(19) = −2.31, p = .032, r = .47. However, analysis of GRS scores did not indicate any between-group differences, t(19) = 0.17, p = .865, r = .04. Similarly, analysis of fabrication time did not indicate any between-group differences, t(19) = −0.25, p = .802, r = .06 (see Figure 2).
Once again, we deconstructed the checklist scores for post hoc analysis of the process and product subscores. Independent t tests of these scores indicated that for the process checklist, there was no difference between groups, t(19) = −0.21, p = .837, r = .05; however, for the product checklist, there was a significant difference between groups, t(19) = −2.59, p = .018, r = .59, whereby the Model Hand group outperformed the Real Hand group (see Figure 3).
Discussion
The purpose of this study was to evaluate an alternative to practicing orthotic fabrication on the hands of classmates. The primary benefit of this approach would be to replicate pathological hand conditions that occupational therapists encounter in practice and therefore better prepare trainees for these scenarios. As a first attempt, however, we used a plastic model of a right hand without any abnormalities and compared it with a real person’s healthy right hand. We used a checklist, GRS, and fabrication time as measures of performance during the course of a 2-hr practice session and on a delayed transfer test.
As we had anticipated, participants improved their performance across the practice sessions. This practice effect was demonstrated by the participants’ ability to create the orthoses more quickly and receive higher scores on global performance, particularly on Trial 3 and onward. Interestingly, participants’ checklist scores did not increase significantly over the course of practice. It is possible that because participants did not receive any external feedback, the particular steps they used to create the orthoses did not change significantly even though they became more comfortable with and fluid in performing the steps over time (as indicated by fabrication time and GRS scores).
During practice, only the checklist indicated a difference between groups: The Model Hand group consistently outperformed the Real Hand group, and this difference was maintained after 1 wk without practice. Because this effect emerged at the beginning of practice, we do not believe that it is solely an effect of enhanced learning and better transfer of skills to create an orthosis on a person’s left hand. Instead, we propose that whatever affected the Model Hand group’s superior performance at transfer is the same factor that affected group differences during practice and, moreover, that this effect of this factor was specifically captured by the checklist.
Our post hoc analyses suggest that the main difference between groups was that the participants using the model hand made a better final product than those using the real person’s hand. To understand this observation, we must consider the functional task difficulty experienced by participants in each group, that is, how challenging a task is in relation to the skill of the performer and the conditions under which the skill is performed (Guadagnoli & Lee, 2004). Because learning is hindered in the presence of too much or too little information, low levels of functional task difficulty optimally challenge novices, who stand to gain much new information from training. Because participants in both groups had similar levels of experience, between-group differences in functional task difficulty can be attributed to task conditions such as simulation fidelity. Chiniara et al. (2012) proposed a framework for simulation in health care in which the fidelity or realism of the simulation is a multidimensional construct that can be characterized by physical fidelity, environmental fidelity, and temporal fidelity (less relevant for our purposes).
If we first consider physical fidelity, it is useful to remember that to properly make an orthosis, the practitioner fits the orthosis on the hand, inspects its fit, makes adjustments, and repeats this process until the correct fit has been achieved. Although the model hand was obtained by a laser scan of a real hand, the model itself was made out of a hard, rigid plastic. Additionally, the fingers and joints were fixed, making the model noticeably different from the hand used by the Real Hand group and by both groups for the transfer test. As such, the physical fidelity of the model could have negatively affected the Model Hand participants’ ability to perform the task by making it more difficult or awkward to achieve a good fit. However, these participants did not need to worry about injuring the hand or inadvertently causing pain; not having these concerns could have made it easier to work with the hand and achieve a good fit. Because these two aspects of physical fidelity (rigidity and insensitivity) have opposing effects on task difficulty, we are inclined to assume that differences in environmental fidelity may have been more relevant to the participants’ performance.
An important aspect of both groups’ performance environment was that they performed the task in the presence of the researcher; however, human interaction was task relevant only to the Real Hand group. This requirement is an element of environmental fidelity and could have added to the Real Hand group’s mental workload: that is, the “relation between the (quantitative) demand for resources imposed by a task and the ability to supply those resources by the operator” (Wickens, 2002, p. 161). Because participants were not explicitly taught how to interact with a simulated client, communication skills and time may have been perceived as limited resources. We observed that participants in the Real Hand group often made comments like, “I’m probably doing such a bad job,” and “I feel like you’re staring at me,” indicating that they felt self-conscious about their performance. Comments also tended to be apologetic, for example, “Sorry, I need your hand one more time.” After the transfer test, one participant in the Model Hand group commented, “I felt like with the model hand, I could just mind my own business and focus on making the orthosis. During the [transfer] test, I felt like you [the researcher] were putting me under the microscope a bit, and it made me a little nervous.”
These anecdotal comments support the notion that the Real Hand group experienced some added stress during practice that the Model Hand group did not encounter until the transfer test. Although this conclusion is speculative, studies in the domain of surgery suggest that simply performing a novel task or a task with which one has little experience can induce stress and that stress impairs performance of technical skills (Arora et al., 2010). Moreover, it has been postulated that moderate stress arouses learners so that their attention is focused on the task and they are able to attend to many cues; however, high arousal restricts learners’ attention, such that vital cues may be missed, resulting in performance decrements (Easterbrook, 1959; Yerkes & Dodson, 1908). This finding suggests that the Real Hand group may have been stressed by performing the task on a person, limiting their attention to performance cues during practice and thereby reducing their learning (performance at transfer). Conversely, participants in the Model Hand group were less stressed during practice and therefore able to hone the method for creating a better orthosis such that their performance was superior even after 1 wk without practice. In summary, we believe that environmental fidelity is the factor that most affected group differences during practice and transfer and that its effect on the quality of the final product was specifically captured by the checklist.
Although simulation is usually meant to replicate a real clinical situation, recent studies on simulation fidelity indicate that simpler environments, lacking some aspects of fidelity, may be adequate for first-time learners (Matsumoto, 2007; Matsumoto, Hamstra, Radomski, & Cusimano, 2002). Brydges, Carnahan, Rose, Rose, and Dubrowski (2010) suggested that learners should be exposed to simulated experiences that progressively increase in fidelity (and functional task difficulty) so that in early stages of learning, trainees can focus on becoming technically proficient, and then in later stages they can incorporate other relevant skills (e.g., communication with the client). These findings provide additional support for the notion of minimizing functional task difficulty for novices by reducing simulation fidelity in the early stages of learning.
Limitations and Future Research
We have demonstrated that having novice occupational and physical therapy students practice making an MCP joint-stabilizing orthosis on a plastic model hand can be part of an effective teaching strategy. However, to maximize utility of such a simulator, we suggest that it be modified to accommodate practice of other types of orthoses. Because the surface of the model hand was very hard and the joints were in a fixed position, it would be difficult to make any orthoses that needed to be slipped over the fingers and impossible to make orthoses that required other hand positions. Nevertheless, this type of simulator may prove to be a powerful teaching tool, especially if the model is used to simulate hand pathologies or as part of a hybrid simulated experience teaching both technical and nontechnical skills.
We took the approach of using a model hand without joint abnormalities to understand whether learning orthotic fabrication was even possible with an artificial hand. Introducing joint deformities at this stage would have made it difficult to determine whether any learning differences were because of the form of the hand or joint deformities. Future work in this area will redesign the model hand to improve physical fidelity and usefulness for multiple types of orthoses and to examine the educational effectiveness of a model hand with arthritic physical characteristics. Additionally, the role of expert feedback during early practice should be studied to determine the type and timing of feedback that facilitates time- and cost-efficient learning gains. Finally, although difficult to implement, retention and transfer tests on a real person with joint deformities would be ideal for understanding the true worth of practicing on model hands.
Implications for Occupational Therapy Practice
The results of this study have the following implications for occupational therapy practice:
Early in training, practicing orthotic fabrication on an artificial hand may have an advantage over a real hand by reducing the additional workload of human interaction, task difficulty, or both, allowing trainees to focus on the technical aspects of the task and become more technically proficient in the early stages of learning.
When developing simulation programs for orthotic fabrication, educators must consider how all aspects of simulation fidelity—physical, environmental, and temporal—may affect trainees’ performance and learning.
Footnotes
Acknowledgments
We thank Patterson Medical for donating the thermoplastic materials we used. This research was funded by the BMO Financial Chair in Health Professions Education Research awarded to Heather Carnahan. The work described in this article was completed as part of Eric Hagemann’s thesis for the degree of Master of Science at the University of Toronto, Toronto.