Date Presented 03/23/24
This presentation reviews the initial psychometric study of inter- and intrarater reliability and content validity for the Visual Activity Sort assessment and intervention, designed to support historically disadvantaged and marginalized youth.
Primary Author and Speaker: Ashley Choi
Contributing Authors: Kathleen O’Day, Sharon Gutman, Sandra Rogers, Alexis Kearney
PURPOSE: The purpose of this study was to establish the content validity, and inter- and intra-rater reliability of the revised Visual Activity Sort, which is a card sorting assessment designed specifically for adolescents and young adults who have been historically disadvantaged and marginalized.
DESIGN: Quantitative mixed methods psychometric study including content experts and youth participants to establish initial interrater & intra-rater reliability & content validity of the Visual Activity Sort.
METHOD: A panel of six content experts was selected and asked to rate the relevancy of each of the 121 Visual Activity Sort cards. Two trained raters participated in the establishment of inter- and intra-rater reliability with 30 high school students attending a charter school for at-risk adolescents and young adults in a socioeconomically disadvantaged urban area.
RESULTS: Based on the content experts’ ratings, a final summary content validity index of 74 maintained cards was established as high (CVI = 0.91, p < 0.05). Both inter- and intra-rater reliability were found to be high with ICCs of 0.83 and 0.84 (p < .000), respectively.
CONCLUSION: Raters and authors recommended the addition of five cards addressing social media and internet safety, setting and maintaining healthy boundaries, managing social media and internet use and addiction, and navigating government agencies. These cards will be added and examined by content experts in a future study.
IMPACT STATEMENT: Many assessment tools derive from a deficit model lens within the medical model. The Visual Activity Sort is a trauma-informed strengths-based tool that has been designed to support historically disadvantaged and marginalized youth. Currently, there are limited occupation-focused assessment tools that are strengths-based and trauma-informed for this population. Research is necessary to further develop strengths-based assessments and interventions for youth in a trauma-informed way.
References
O’Day, K. (2020). Visual Activity Sort. https://www.visualactivitysort.com/
Gutman, S., O’Day, K, Rogers, S.. AOTA Inspire 2023. In Examining & Selecting Assessments that are Inclusive of Diverse Service Recipients. Kansas City, MO.
Gutman, S., O’Day, K., Choi, A., Rogers, S., & Kearney, A. (n.d.). Establishing Initial Content Validity, Interrater Reliability, and Intra-rater Reliability of the Revised Visual Activity Sort for At-Risk Adolescents and Young Adults (in press). Open Journal of Occupational Therapy.
