Measuring Recovery in Mental Health: Why the Next 100 Years of Occupational Therapy Practice Depends on Metrology and Meaningful Measurement

PURPOSE: Across the globe, there is an increasing commitment to personal recovery as the desired outcome for mental health rehabilitation, yet there is little agreement about how to measure it (World Health Organization, 2013). Recovery is defined as “a way of living a satisfying and contributing life, even with the limitations caused by illness” (Anthony, 1993). In order for occupational therapists to deliver evidence-based recovery-based services, a robust clinically meaningful measure is needed. The purpose of this study was to develop a measure of personal recovery for use by people with mental illness living in the community.

DESIGN: This study was a mixed-methods study with five phases. We used an iterative process of quantitative and qualitative methods to generate a set of items that covered the full range of personal recovery for people with mental illness.

METHOD: In Phase 1, 228 adults (M age = 45 yr, 55% women) in a large city completed six recovery questionnaires. We used traditional and Rasch measurement methods (Andrich, 2011) to evaluate the extent to which these questionnaires were fit for purpose to measure the full range of recovery. In Phase 2, we conducted two focus groups with 20 adults with schizophrenia to review results in Phase 1, generate new items to measure recovery, and propose a hierarchy of items that could cover the full range of recovery from low to high. In Phase 3, 299 adults with mental illness completed the new items (M age = 44 yr, 51% women), and we used similar methods from Phase 1 to evaluate the reliability and validity of the item set. In Phase 4, we held another focus group with 10 adults to review results and explain anomalies in the results. In Phase 5, we administered the new item set to 575 individuals and presented results to eight expert groups in four cities (total of 250 clinicians, people with lived experience, and family members) to receive feedback on the final item set, item hierarchy, clinical applicability, and utility of the tool across a full range of community mental health services.

RESULTS: In Phase 1, no single questionnaire met the criteria for robust measurement outlined by traditional or Rasch methods. In Phase 2, 40 new items were generated. In Phase 3, analysis of these items showed good overall fit to the Rasch model, χ²(62) = 76.3, p = .11, misfit of three of 40 items, high reliability (r _p = .94), an ordered response scale structure, and no item bias for gender, age, or time. In Phase 4, participants identified that the scale was too long, and a shortened 30-item set was agreed upon. This item set was labelled the Personal Recovery Outcome Measure (PROM). In Phase 5, the PROM showed good overall fit to the Rasch model, χ²(130) = 163, p = .05, no item misfit, high reliability (r _p = .92), an ordered response scale structure, and no item bias for gender, age, diagnosis, or time. The hypothesized hierarchy held for all items. Expert consultation suggested that the PROM was clear and provided a common language for comparison between groups and within individuals. The PROM was described as a means to monitor recovery outcomes and have a systematic conversation about recovery goals and the services.

CONCLUSION: This study supports a new 30-item measure of recovery as defined by people with mental illness. The measurement model underpinning the PROM has potential to support the clinical relevance of recovery scores for occupational therapy, thereby advancing an evidence-based approach to mental health rehabilitation and a common language for assessment, treatment planning, and program evaluation.

References

Andrich, D. (2011). Rating scales and Rasch measurement. Expert Review of Pharmacoeconomics and Outcomes Research, 11, 571–585. https://doi.org/10.1586/erp.11.59

Anthony, W. A. (1993). Recovery from mental illness: The guiding vision of the mental health service system in the 1990s. Psychosocial Rehabilitation Journal, 16(4), 11–23. https://doi.org/10.1037/h0095655

World Health Organization. (2013). Mental health action plan 2013–2030. Geneva: Author.