Relative Mastery Measurement Scale as an Effective,Profession-Specific,Theoretically Founded Learning Outcome Measure

PURPOSE AND BACKGROUND: The purpose of this study was to evaluate the psychometric properties of the revised Relative Mastery Measurement Scale (RMMS). The primary aim was to evaluate the sensitivity of the RMMS to change over time within an academic setting. The secondary aim was to assess learner perceptions of relative mastery.

The RMMS is an assessment instrument based on the theory of Occupational Adaptation (Schkade & Schultz, 1992; Schultz & Schkade, 1992). The theory proposes that people desire to do well in their daily performance and assess it on the basis of a review of their relative mastery. Relative mastery includes a sense of effectiveness, efficiency, and satisfaction to self and others. George and colleagues (2004) developed the RMMS as a valid and reliable means to measure relative mastery. The original instrument contained 27 items worded positively and negatively addressing each element of relative mastery. Rasch analysis confirmed the scale as unidimensional and as having construct and content validity. The authors revised the RMMS to resolve two misfitting items and reduce the overall number of items; the revised RMMS was used in the current study.

Garrett and Schkade (1995) posed Occupational Adaptation as a framework to consider learner adaptive responses. No other OA assessment is currently in use for that purpose. Hooper et al. (2013) conducted an international systematic mapping review of education approaches and teaching methods. Among other findings, they identified current educational intervention as addressing learner “skills, knowledge, attitudes and perceptions, but not their behaviours, organisational change or impact on clients” (p. 9). Further, they identified as a priority the need to develop “profession-specific conceptual frameworks for learning and measure outcomes beyond student views of the learning activity” (p. 9). We applied profession-specific OA theory with and for learners to assess their development of relative mastery, an outcome beyond their view of the activity.

METHOD: The study used a mixed methods design. Quantitative descriptive methods provided systematic information about the data, and qualitative narrative inquiry uncovered learner experience. Participants were 39 entry-level doctorate occupational therapy students in one course at one university. We collected data as part of routine self-assessment for learning. We viewed the data retrospectively as part of course and instructor evaluation. The institutional review board exempted the study from review.

The revised RMMS was used to collect data from learners three times during the course. We collected numeric scores and digitally written responses to open-ended questions. The primary aim was to evaluate the sensitivity of the RMMS to change over time. We analyzed useable quantitative data from 33 participants through repeated measures analysis of variance.

RESULTS: Analysis revealed a significant effect of relative mastery across time, F(2, 84) = 44.26, p < .0001. To assess learner perceptions of relative mastery, we reviewed narratives of all 39 participants. From narrative thematic analysis emerged participants’ adaptive strategies to achieve relative mastery during preparation for an Objective Structured Clinical Examination. Learners commented on their process of change across effectiveness, efficiency, and satisfaction. Learners voiced frustration and stress as prompts for future growth, the foundation of the concept of negative relative mastery.

CONCLUSION: The results of the study are useful for education and practice. The study supports the RMMS as a profession-specific learning outcome measure. The study describes the effectiveness of an instrument founded on an occupational therapy theoretical frame of reference. Findings suggest the RMMS to be sensitive to measure change over time. Analysis showed a significant trend indicating utility for learners to assess their developing sense of mastery. Narrative analysis suggested usefulness of the RMMS as a prompt for adaptation. We are applying the instrument in other settings with other client populations. The instrument is undergoing further modification to reduce the number of items to six and to revise the agree–disagree scale to an item-specific scale.

References

Garrett, S. A., & Schkade, J. K. (1995). Occupational Adaptation model of professional development as applied to Level II fieldwork. American Journal of Occupational Therapy, 49, 119–126. https://doi.org/10.5014/ajot.49.2.119

George, L. A., Schkade, J. K., & Ishee, J. H. (2004). Content validity of the Relative Mastery Measurement Scale: A measure of occupational adaptation. OTJR: Occupation, Participation & Health, 24, 92–102. https://doi.org/10.5014/ajot.46.10.91710.1177/153944920402400303

Hooper, B., King, R., Wood, W., Bilics, A., & Gupta, J. (2013). An international systematic mapping review of educational approaches and teaching methods in occupational therapy education. British Journal of Occupational Therapy, 76, 9–22. https://doi.org/10.5014/ajot.46.10.917

Schkade, J. K., & Schultz, S. (1992). Occupational Adaptation: Toward a holistic approach for contemporary practice, Part 1. American Journal of Occupational Therapy, 46, 829–837. https://doi.org/10.5014/ajot.46.9.829

Schultz, S., & Schkade, J. K. (1992). Occupational Adaptation: Toward a holistic approach for contemporary practice, Part 2. American Journal of Occupational Therapy, 46, 917–925. https://doi.org/10.5014/ajot.46.10.917