Examining the Predictive Validity of the Health Sciences Reasoning Test (HSRT) in a Nontraditional Graduate OT Program

PURPOSE: The purpose of this study was to determine if the HSRT predicts success within a nontraditional graduate OT program. The researchers hypothesized that the HSRT is a valid predictor of student success as they enter and progress through a graduate OT program. The independent variable was HSRT score and the dependent variable was student success as measured by successful completion of the program within the expected time sequence. The investigated problem was that there is no specific valid measure to predict student success in the admission process. This problem needs a solution because 17% of students are not completing the program in the expected sequence. Thus, examining HSRT scores prior to admission could provide predictive evidence of student success.

DESIGN: This study used a quantitative non-experimental ex post facto design to evaluate the predictive ability of the HSRT on student success in one group of students enrolled in a non-traditional graduate OT program between years of 2011 and 2015, in Blauvelt, New York. The sample of student participants was recruited through non-random convenience sampling as all students’, from 2011 to 2015, HSRT scores were examined and readily accessible. Participants met the inclusionary criteria if they were accepted and currently enrolled in graduate level coursework at the participating college. To increase the rigor of the study, the researchers excluded participants who left the program for personal reasons. This ensured that the data collected solely represented participants who did not successfully complete the program due to academic reasons. The HSRT was implemented in a group setting where each student had access to his/her own desktop computer and took the HSRT individually during the program’s orientation day. The computer issued HSRT web-based program produced scores based on students’ responses during the multiple-choice evaluation.

METHOD: The data was collected and de-identified by the OT program director to ensure confidentiality prior to statistical analysis. Primary data analysis was assessed using an ordinal logistic regression and a Spearman rho correlation coefficient to determine the prediction and the relationship between HSRT scores and student success.

RESULTS: The sample size consisted of 209 students. An ordinal logistic regression was applied to determine if HSRT scores predict student success. A significant regression equation was found (F1, 209) = 16.287 p<.001), with a Nagelkerke R² of .089, p<.05. For every increase in HSRT score, a students’ likelihood of being successful increased by .051. A Spearman rho correlation coefficient was used to determine if there was a correlation between HSRT scores and student success. A statistically significant, weak, positive correlation was found (rho (209) = .246 p<.001). The results indicated that the HSRT is not a valid predictor of student success. There is also a weak relationship between HSRT scores and student success.

CONCLUSION: The study found that the HSRT is not a valid predictor of graduate OT student success. The results can be used to further the OT profession. Researchers can study whether the HSRT along with other components such as undergraduate GPA, on-site essay score, and on-site interview score better predict student success. They can also determine if any of the five components of critical thinking accurately measure student success rather than overall HSRT score. Lastly, researchers can evaluate if lower scores on the HSRT are correlated with higher rates of failure. This will assist admissions counselors in selecting students who are likely to be successful in a graduate OT program.

References

Al Alwan, I., Al Kushi, M., Tamim, H., Magzoub, M., & Elzubeir, M. (2013). Health sciences and medical college preadmission criteria and prediction of in-course academic performance: a longitudinal cohort study. Advances in Health Sciences Education: Theory and Practice, 18, 427-238. https://doi.org/10.1007/s10459-012-9380-1

Brudvig, T., Mattson, D. J., Guarino, A. J. (2016). Critical thinking skills and learning styles in entry-level doctor of physical therapy students. Journal of Physical Therapy Education, 30(4), 3-10. Retrieved from https://web-a-ebscohost-com.libdb.dc.edu/ehost/pdfviewer/pdfviewer?vid=4&sid=1d895274-2df0-47f6-8018-af247018730e%40sessionmgr4008

Cox, W. C. & McLaughlin, J. E. (2014). Association of health sciences reasoning test scores with academic and experiential performance. American Journal of Pharmaceutical Education, 78(4), 1-73. doi: 10.5688/ajpe78473

Facione, N. C., & Facione, P. A. (2008). Critical Thinking and Clinical Reasoning in the Health Sciences: A multidisciplinary teaching anthology. Millbrae, CA: The California Academic Press