Development and Psychometric Testing of the Functional Simulated Technology Tasks

BACKGROUND AND PURPOSE: In 2010, nearly 80% of adults used the Internet (U.S. Census Bureau, 2011). The increase in technology use by all generations increases the demand for occupation-based technological assessments and interventions. To best understand occupational engagement and formulate appropriate treatments, top-down assessments are needed to adequately assess clients’ use of technology, which requires executive function skills. Currently available executive functioning assessments often evaluate paper bill payment and check writing and telephone use but fail to address technology use.

The Functional Simulated Technology Tasks (FSTT) is a performance-based measure of executive function that includes an online bill pay activity and an online shopping activity (Cardell et al., 2013). The FSTT’s format and scoring are modeled after the Performance Assessment of Self-Care Skills (Rogers & Holm, 1994). The purpose of this study was to evaluate the FSTT’s construct validity. The specific research question was, Do people identified as having cognitive impairment score lower on the FSTT than people without cognitive concerns?

METHOD: Construct validity was evaluated through two approaches: evaluation of the relationship between FSTT Outcome and Independence scores for the clinical and control groups and correlation between Montreal Cognitive Assessment (MoCA; Nasreddine et al., 2005) scores and the FSTT scoring areas. Descriptive statistics were used to characterize the performance scores. Comparison between groups was performed with nonparametric Mann–Whitney U tests. Spearman rho tests were used to further examine correlations between the two groups. Significance was set at p ≤ .05.

Participants were community-dwelling adults with and without cognitive impairment recruited from a convenience sample. Participants’ mean age was 42.6 (range = 25–76), and identified diagnoses varied. Participants who scored ≤25 (range = 12–25) on the MoCA (n = 20) were placed in the clinical group, and those who scored >26 (range = 26–30; n = 20) were placed in a control group matched on age, gender, and education.

RESULTS: Mean rank scores were significantly higher for the control group than for the clinical group in all scoring areas. To further examine the relationship of the scores between groups, a correlation coefficient test was performed between MoCA scores and the two score areas of each FSTT task. Results indicate a good relationship between scores on the MoCA and FSTT (rs = .662–.778, p < .01), indicating that as scores on the MoCA increase, so do scores on the FSTT.

CONCLUSION: Occupational therapy evaluations include occupation-based performance assessments. Given the frequent use of technology, there is a need to update occupational therapy assessments to reflect up-to-date occupations. Although the psychometric properties of the FSTT require further evaluation under more rigorous conditions, these results show preliminary support for the validity of the tool for use as a performance-based executive function assessment.

References

Cardell, B., Swain, L. J., & Burnett, A. (2013). Construct validity of the Functionally Simulated Technology Task: An exploratory study. Occupational Therapy in Healthcare, 27, 345–354. https://doi.org/10.3109/07380577.2013.845928

Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., . . . Chertkow, H. (2005). The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53, 695–699. https://doi.org/10.1111/j.1532-5415.2005.53221.x

Rogers, J. C., & Holm, M. B. (1994). Performance Assessment of Self-Care Skills [Unpublished assessment tool]. Pittsburgh, PA: University of Pittsburgh.

U.S. Census Bureau. (2011). Statistical abstract of the United States: 2011. Retrieved from https://www.census.gov/library/publications/2010/compendia/statab/130ed.html