Abstract
Educational requirements for occupational therapy practitioners are evolving, and although a master’s degree currently satisfies the requirements for entry-level practitioners (American Occupational Therapy Association [AOTA], 2019), the number of new and transitioning entry-level doctoral programs is growing rapidly (Accreditation Council for Occupational Therapy Education, 2019). For occupational therapy educators, the increased focus on education signals a demand for advanced content and teaching knowledge (Brown et al., 2015).
Occupational therapy educators are also being charged with leveraging educational technology to create robust educational experiences for digitally native students (Guze, 2015). As educators strive to meet the challenges of revising curricula and refining instructional practices, the need for research on effective educational practices grows. Schaber (2014) encouraged occupational therapy educators to become scholars of teaching and learning. The revised occupational therapy educational research agenda (AOTA, 2018) offers a structure for investigating teaching and learning issues and recommends a focus on theory building, pedagogy, and instructional methods. Because occupational therapy educators are typically seasoned content experts, these guidelines signal them to expand pedagogical, instructional, and technological research to adapt to the changing landscape.
As educational and accreditation requirements evolve, expectations placed on entry-level clinicians will also intensify. Students will need to demonstrate an understanding of more complex conceptual and theoretical topics and apply that knowledge to clinical scenarios. The prevalence of digital technology in educational and work environments will also mandate increased sophistication and competencies.
Navigating these changes in occupational therapy education will necessitate insight into learning processes, the foundation of which can be found in contemporary learning theories involving meaningful and transferable learning (Ausubel et al., 1978) and the physiological underpinnings of human cognition and learning (Banich & Compton, 2018). Educational psychologists advocate for balanced emphases on content and pedagogical knowledge (Shulman, 1987) and mindful integration of technology in didactic and clinical experiences (Koehler & Mishra, 2009). Investigating how learners process information is informed by cognitive load theory (CLT; Moreno & Park, 2010), which describes the intersections among cognitive architecture, instructional design, and learner characteristics as they affect the educational process. The cognitive theory of multimedia learning (CTML; Mayer, 2014) provides a set of evidence-based practice principles that guide presentation design for more effective learning (Table 1).
Elements of Traditional Enhanced Pretraining Following Cognitive Theory of Multimedia Learning Principles
Source. Mayer and Pilegard (2014).
Both CLT and CTML classify three types of mental energy that affect learning (Sweller et al., 2011). Extraneous processing results from presentations with poor design (e.g., distracting layouts or “wall of text” slides combined with concurrent narration). Essential processing involves the inherent difficulty of the information to be learned. Generative processing concerns the integration of new and prior knowledge and moving information into long-term memory (Mayer, 2014).
The current study focuses on essential processing and examines complex content learning within occupational therapy education. The study examines the pretraining principle, which indicates that “people learn more deeply from a multimedia message when they know the names and characteristics of the main concepts” (Mayer & Pilegard, 2014, p. 316). The challenging topic of sensory integration (SI) theory was used for this research.
Pretraining addresses complex content, defined as any topic with a high level of element interactivity—that is, many items of information that must be processed simultaneously in working memory to understand the given topic (Sweller et al., 2011). Working memory functions to briefly store information as it is being manipulated during the performance of complex tasks (Baddeley, 2015). When too many items must be understood concurrently, working memory becomes overloaded. The learner’s level of prior knowledge determines how difficult a topic is to learn, and information that is not actively manipulated in working memory is quickly forgotten.
Pretraining traditionally involves construction of a two-stage mental model that encourages engagement with information and schema building (Mayer et al., 2002). First, the learner is introduced to concrete terms pertaining to a topic. Next, content with a higher degree of element interactivity is presented. Pretraining thus involves a foundational activity that is followed by a presentation of more conceptual and nuanced content containing highly interactive elements.
Pretraining is most effective with learners who have a low level of prior knowledge, fast-paced instruction, and topics with a high level of complexity (Mayer, 2009). A solid body of evidence supports the use of pretraining (Clarke et al., 2005; Pollock et al., 2002). In addition, 13 of 16 identified pretraining experiments yielded a medium-high median Cohen’s d of 0.75 (Mayer & Pilegard, 2014).
This research examined approaches that augment traditional pretraining—in particular, the addition of a concept map. The concept map is a visual model composed of nodes containing a word or phrase and links that connect ideas to one another. Concept maps are powerful tools for integrating information because they provide an overview of a topic while highlighting relationships among the presented facts (Schraw & Paik, 2013). Adding a concept map to pretraining has been explored only minimally (Cutrer et al., 2011), yet this strategy shows significant potential benefit because the learner could theoretically build foundational knowledge while previewing upcoming complex content.
In this study, two types of concept maps were compared with traditional pretraining to examine their effect on achievement among occupational therapy students. The first was a static concept map presented in its entirety on a screen. The second was an animated concept map, which presents information gradually through a series of unfolding images delivered in a video format. In part, animated concept maps were developed to address the cognitive overload—often called map shock (Blankenship & Dansereau, 2000)—that can result from viewing a heavily detailed static concept map. A complex static concept map paired with a lack of navigation strategy to guide the viewer can make the map difficult to comprehend. Theoretically, animated concept maps lessen stress on working memory by sequencing information (Nesbit & Adesope, 2013).
The development of the pretraining intervention modules for this study involved augmenting traditional pretraining with multimedia elements such as visual and auditory cuing, spatial arrangement on slides, and limited on-screen text (Mayer, 2009). The resulting novel approach to pretraining is referred to here as traditional enhanced pretraining, and the strategies used, following CTML best practice principles, are detailed in Table 1. Despite the changes made to traditional pretraining, it was hypothesized that the sequencing used in an animated concept map would result in the greatest gains in achievement.
Method
Participants
First-year students (N = 145) were recruited from three occupational therapy programs and included 75% entry-level master’s students, 13% entry-level doctoral students, and 12% bachelor’s-to-master’s students. The sample was 85% female, and 75% of participants were ages 20 to 29 yr. The majority of participants (87%) had one or fewer experiences with SI theory.
Materials
Participants were exposed to one of three types of pretraining intervention. All groups then received the same multimedia lecture. Pretraining videos introduced participants to key SI terms detailed in Figure 1. The narrated videos contained identical content except that signaling phrases were used in the static concept map module (e.g., “On the left side of your screen . . .”). The traditional enhanced pretraining module included slide images augmented by limited on-screen text. The static concept map pretraining module consisted of a one-screen map, and the animated concept map pretraining module used the static concept map as a template but gradually revealed individual terms until the map was viewed in its entirety.
Sensory integration theory concept map used for pretraining.
All pretraining modules adhered to CTML principles (Mayer, 2014), and the map modules followed best practices for creating concept maps (Nesbit & Adesope, 2013). Pretraining videos can be viewed at Kaye (2018).
The final lecture content on SI theory was developed from a thorough review of the literature (e.g., Ayres, 1979; Lane et al., 2014; Parham & Mailloux, 2014), reviewed by several content experts, and revised three times. The presentation was created using best practices from the CTML (see Table 1). Design guidelines regarding use of color, space, text, and images were also used (Reynolds, 2014). A detailed script and use of the same instructor for all lessons ensured that the lecture was consistent across all presentations.
Pretraining achievement was focused on schematic knowledge acquisition at pretest, posttest, and delayed posttest. Schematic knowledge refers to the web of interrelated information built during the learning process. Schemata link new information to relevant prior knowledge and help move information from working to long-term memory. Measuring schema formation rather than retention emphasizes the level of learning necessary for subsequent problem solving and transfer.
Measure
Schematic knowledge was measured with a 5-item structured word association test that quantified cognitive organization of topical content (Shavelson, 1974). Participants were given 1 min per item to write down as many terms related to a target term as possible. Terms were representative of core SI concepts and included proprioception, vestibular, praxis, modulation, and sensory integration. A key developed from content experts’ review and pilot testing was used to score the subtests. Test validity was examined with Lawshe’s content validity index, which provides a statistical means for content experts to validate an assessment instrument through item review (Gilbert & Prion, 2016). The overall content validity index for the test instrument was .90, a strong score. Internal item consistency was also calculated, and Cronbach’s α was acceptable at .77.
Design
The quasi-experimental pretest, posttest, and delayed posttest comparison study consisted of a 12-min pretraining intervention video followed by a 60-min live multimedia lecture on SI theory. Institutional review board approval was secured from all participating institutions, and written consent was obtained from each participant.
Procedure
In lieu of their regular weekly lesson, participants were presented with one of three pretraining intervention modules: traditional enhanced pretraining, pretraining with a static concept map, or pretraining with an animated concept map. Intervention type was randomly assigned to intact class sections. All participants then received the SI lecture and the posttest. Digital devices were not permitted during the session, but participants could take handwritten notes.
Participants were provided with a one-page handout of the pretraining content. The traditional enhanced group received a list of the pretraining terms, and the two concept map groups received a black-and-white concept map. Participants were instructed to refrain from discussing or studying SI until after data collection was complete. Two weeks after the intervention, a delayed posttest was administered. A sampling of random tests was rescored to ensure scoring accuracy.
Data Analysis
The scores on the pretest, posttest, and delayed posttest were compared to measure change. Repeated-measures analyses of variance (ANOVAs), which allow for the analysis of treatment groups independently of each other and account for the compounding error resulting from testing groups repeatedly over time, were used to assess participant achievement. After the primary analysis, an ancillary analysis of a matched trio subsample of participants with the lowest prior knowledge levels was completed to further investigate the findings.
Results
Primary Analysis
Because the treatment groups were not equivalent at pretest, the pretraining treatment effect was measured through gain—that is, growth over time—for each of the three groups. Historically, this method has been criticized for reliability and error rate issues (Cronbach & Furby, 1970), but Rogosa and Willett (1983) challenged this argument and showed that gain scores can provide a valid method of measuring change over time and an unbiased estimate of the population change score. Mauchly’s test of sphericity was completed to verify the assumption of homogeneity of variance (Portney & Watkins, 2015). The results were not significant at p < .05, indicating that the repeated-measures ANOVA outcomes provided a valid analysis of achievement over time. In addition, because each group included more than 30 participants, the groups can be assumed to have a multivariate normal distribution.
Gain scores from pretest to posttest indicated statistically significant differences between the treatment groups, F(2, 142) = 3.67, p < .05, as summarized in Table 2. Post hoc analysis revealed that the mean gain scores were higher in the static concept map condition than in the traditional pretraining or animated concept map conditions. No statistically significant differences were found between the traditional pretraining and animated concept map groups. The partial η2 effect size was .05, a small to medium effect. Despite the greater overall decline in scores for the static concept map group at delayed posttest, gain scores for the three groups from pretest to delayed posttest were comparable to each other (Figure 2).
Schematic Knowledge Gain Scores for Primary and Ancillary Analysis of Traditional Enhanced, Static Concept Map, and Animated Concept Map Pretraining
Note. ACM = animated concept map pretraining; SCM = static concept map pretraining; TP = traditional enhanced pretraining.
Statistically significant at .05 while controlling for overall error rate.
(A) Primary and (B) ancillary analysis: Gain scores at posttest and delayed posttest.
Ancillary Analysis
To further examine the findings, a matched trio subsample of participants with the lowest prior knowledge levels (n = 45) was identified and analyzed. Because pretraining is most effective for learners with low prior knowledge of a topic and the effect of pretraining diminishes as learners’ knowledge base increases (Mayer & Pilegard, 2014), the subsample was created to examine these prime candidates and to compare findings using a sample with pretest group equivalence. Matched trios were identified on the basis of pretest scores, prior exposure to the topic of SI, and type of degree program.
The ancillary results, detailed in Table 2, reinforced the primary findings and indicated that the static concept map group had the greatest gains in scores from pretest to posttest. At delayed posttest, each group experienced an expected decline in scores, yet the static concept map group continued to demonstrate the strongest scores (Figure 2).
Discussion
All groups showed achievement gains after receiving pretraining to support learning about SI theory. These results present support for the use of pretraining to teach topics in occupational therapy education and as an effective preparatory learning strategy. The ancillary findings support the primary results and provide further perspective on the effectiveness of pretraining for learners with the lowest level of prior knowledge.
Traditional Enhanced Pretraining
The findings in the primary and ancillary analyses show positive mean gains and retained knowledge gains, suggesting that the use of traditional enhanced pretraining provides an effective approach for learning. The traditional enhanced module expanded on learner engagement through the addition of multimedia techniques (Mayer, 2010). Of particular note was the use of visual signaling to guide the learner’s attention to the material’s most salient features and organization (Mayer & Fiorella, 2014). This design also limited the presentation to the use of images and narration rather than abundant on-screen text. Whether traditional enhanced pretraining is more effective than traditional two-stage pretraining is a question for further study.
Pretraining With a Static Concept Map
In the primary analysis, the static concept map group started with the lowest pretest scores, potentially because this was the only group composed in part of bachelor’s-to-master’s students. This group had the strongest gains from pretest to posttest, and despite a commensurate retention of learning, it also demonstrated the largest declines after 2 wk, at delayed posttest. In the ancillary analysis of the learners with the lowest level of prior knowledge, the static concept map group again demonstrated the strongest gains at posttest and, surprisingly, the least decline at delayed posttest. In fact, the static concept map group demonstrated stronger scores at delayed posttest than the other two groups had demonstrated immediately after the intervention, at posttest.
The contradictory results between the primary and ancillary analyses suggest that the concept map preferentially assisted learners with low prior knowledge, yet it did not translate into retention at delayed posttest unless the learners had extremely low prior knowledge. It is unclear whether the participants’ abilities, the static concept map, or the interaction of the two accounted for the drop in retention scores. In addition, although the ancillary findings were not statistically significant—probably as a result of the small sample size—the persistently strong scores at delayed posttest provide support for the assertion that learners with the lowest prior knowledge benefit the most from pretraining. Areas for future research include interactions between prior knowledge and static concept maps as well as strategies for cementing retention of knowledge.
Pretraining With an Animated Concept Map
Contrary to expectations, use of the animated concept map did not result in superior outcomes in either the primary or the ancillary analysis. These results were unexpected because of the current propensity for technology-enhanced educational materials and support for the use of animated concept maps (Nesbit & Adesope, 2006). Although animated visuals can be effective learning resources, the transient nature of the images might produce cognitive overload (Schnotz & Rasch, 2005). This paradoxical effect intensifies for learners with low prior knowledge because the novelty of animation adds to an already increased learning load and overloads working memory capacity (Höffler & Leutner, 2007). The narration may also have decreased the impact of the animated concept map because it aided learners in decrypting the static concept map and thus lessened the differentiation between the two resources (Adesope & Nesbit, 2013).
A recent meta-analysis of concept maps found that animated maps are best used with procedural rather than semantic topics and raised questions about the superiority of animated maps over static ones (Schroeder et al., 2018). In addition, substantial amounts of time and technical knowledge are needed to create animated materials, so instructor resources might be better spent elsewhere.
Limitations and Future Research
Knowledge gains from classroom research are valuable, yet research in the natural setting presented limitations as a result of lack of random group assignment and limited controls. The demographic profile of participants and small subsample sizes presented additional limits to the generalizability of the study. Future research directions should include use of pretraining as part of online preparation and further investigation into augmenting pretraining with visual models such as concept maps.
Implications for Occupational Therapy Education
The findings of this study have the following implications for occupational therapy education:
Pretraining is an effective instructional strategy for use with fast-paced instruction, complex content, and students with low prior knowledge.
Using a static concept map and leveraging multimedia-based instruction to present pretraining resources can enhance achievement.
A document with which to review pretraining information augments the strategy and provides a valuable study guide.
Knowledge and use of CLT, CTML, and presentation design principles benefit educators as they develop, implement, and evaluate classroom instructional strategies.
Conclusion
Occupational therapy educators can enhance teaching effectiveness by expanding the content-rich, hands-on kinesthetic legacy of instruction to encompass pedagogical and technological resources. Educators working to optimize learning processes will find that multimedia techniques provide practical and accessible instructional road maps. Pretraining and related techniques offer potent tools to assist learners in accessing and understanding complex content. Students can then focus their efforts beyond fact retention and toward problem solving and skill transfer. Developing these robust abilities will fortify health care students to meet the increasing rigors of academic training and to more competently transition to professional practice.
Footnotes
Acknowledgment
This research was conducted at the University of San Francisco, San Francisco, CA.