Abstract
Ashwini K. Rao, EdD, OTR/L, FAOTA
To support the continued role of occupational therapy in the changing landscape of health care policy, the challenge for practitioners is to develop assessment tools that are valid, reliable, and sensitive to change in performance; to develop and test functional interventions that demonstrate improvements at the impairment, activity, and participation levels; and to develop methods to improve adherence to therapeutic interventions. The focus of this editorial is on choosing assessment tools and measuring function in chronic progressive disorders.
Challenge of Chronic Progressive Disorders
The U.S. population is aging. According to the U.S. Census Bureau (2012), life expectancy was 78.0 yr in 2008 and is projected to reach 79.5 yr by 2020. As the population ages, the prevalence of chronic diseases and disability will increase (Crimmins & Saito, 2000). Even in middle age, the proportion of people who require assistance with activities of daily living increased from 1997 to 2007 as a result of chronic diseases (Martin, Freedman, Schoeni, & Andreski, 2010). Chronic progressive disorders are particularly challenging because the underlying pathology and body functions worsen with time, resulting in greater activity limitations and disability (Murphy, 2010).
Occupational therapy practitioners routinely work with people who have chronic neurological disorders, including Parkinson’s disease (Rao, 2010), Huntington’s disease (Bilney, Morris, & Perry, 2003), multiple sclerosis (Steultjens, Dekker, Bouter, Leemrijse, & van den Ende, 2005), amyotrophic lateral sclerosis (Lewis & Rushanan, 2007), essential tremor, and Alzheimer’s disease (Letts et al., 2011). For example, in Parkinson’s disease, the primary pathology is a depletion of dopamine in the substantia nigra, a nucleus in the basal ganglia. Dopamine depletion leads to motor and nonmotor impairments that give rise to activity limitations and participation restrictions. As the disease progresses, impairments and resulting activity limitations worsen (Evans et al., 2011).
Figure 1 presents a simplified hypothetical scenario in which functional decline takes place over time. Decline in functional ability beyond a critical point results in disability. Occupational therapy services aim to modify the slope of functional decline so that the onset of disability is either delayed or prevented. It is therefore important to choose assessment tools that are responsive to change in performance.
Choosing Assessment Tools to Measure Changes in Function
Occupational therapists are skilled at measuring body function, activity limitations, and participation restrictions (World Health Organization [WHO], 2001). Doucet and Gutman (2013) recommended the use of multiple assessment tools that provide information on body function, activity limitations, and participation. The following sections briefly discuss considerations in the selection of assessment tools on the basis of the framework proposed by Law and MacDermid (2008).
Focus of Measurement
The International Classification of Functioning, Disability and Health (ICF; WHO, 2001) provides a framework for evaluating whether assessment tools measure body function, activity limitation, or participation restrictions. For instance, balance and mobility can be measured using assessments of body function (e.g., Berg Balance Scale [BBS]; Berg, Wood-Dauphinee, Williams, & Maki, 1992) or activity limitation (e.g., Timed Up and Go [TUG] test; Podsiadlo & Richardson, 1991).
Measurements can be used to describe current function (descriptive), predict function at a later time (predictive), or document change over time (evaluative; Law & MacDermid, 2008). In clinical practice, evaluative assessments are perhaps most commonly used because they allow practitioners to measure change following intervention. For example, to measure balance in Parkinson’s disease, assessment tools may include the BBS, Functional Reach Test, and TUG test (Steffen & Seney, 2008). These scales were developed specifically to measure change in function.
Clinical Utility
The clinical utility of an assessment tool is enhanced if the instructions for test administration are clear and if the test can be administered in a reasonable period of time. In addition, the tool must be freely available to practitioners across practice settings.
Scale Construction
Practitioners need to determine whether the assessment tool includes all relevant items pertaining to the function being tested. For instance, in assessing balance, it is important to use an assessment tool that tests both prospective and reactive balance within a functional context. It is also important to consider the level of measurement used in the assessment tool, which may be nominal (descriptive categories; e.g., male and female), ordinal (ordered categories; e.g., minimal, moderate, and maximal assistance), or ratio scale (interval scale with a true zero point; e.g., time spent in one leg stance). Assessment tools that use a ratio scale are best suited for measuring functional change over time.
Standardization
Practitioners should choose assessment tools with specific criteria for administration, scoring, and interpretation. Standardized assessments enable comparison of assessment scores within a person at different times and across individuals. Clear standards for test administration hold every client and tester to the same criteria, consistency that is important in interpretation of test results.
Characteristics of an Assessment Tool
Validity.
Validity indicates whether an assessment tool measures what it claims to measure. Validity is typically established by comparing the assessment tool with a gold standard tool (Law & MacDermid, 2008). When choosing an assessment tool, practitioners should examine whether it has been validated for the population in question.
Responsiveness.
Responsiveness is the ability of an assessment tool to measure change in function over time. Responsiveness can be evaluated in terms of minimal detectable change, minimal clinically important change, and item response theory.
Minimal detectable change (MDC) indicates whether change in function reflects actual change that is greater than random variation in measurement (Haley & Fragala-Pinkham, 2006). MDC is calculated from the standard error of measurement, a confidence factor of change in measurement, and test–retest reliability. In documenting change in function, it is important to ensure that the observed change is greater than random variation. For example, for the BBS, the MDC for Parkinson’s disease is 5 points (Steffen & Seney, 2008). Any change in score less than 5 points may not be considered a true change in function.
Minimal clinically important change (MCID) refers to the minimal change in score that is clinically meaningful to individual clients. Because MCID is change that is important to clients, assessment may involve asking clients to evaluate whether the change in function is meaningful in their daily life.
Scores on assessment tools developed using item response theory (IRT) methodology take into account the relative difficulty of individual items in the tool. For example, the Manual Ability Measure is a client-centered functional questionnaire that assesses hand function (Chen, Granger, Peimer, Moy, & Wald, 2005). The advantage of using such an instrument is that a score is anchored to the client’s actual ability; when the client’s ability improves, the test score changes accordingly. Assessment tools designed with this methodology thus provide information on clinically important change.
Reliability.
Reliability refers to the consistency between repeated measurements made by an assessment tool when there has been no change in function. Three forms of reliability are test–retest, interrater, and internal consistency.
Test–retest reliability refers to consistency in test results when the assessment tool is administered on more than one occasion. For instance, test–retest reliability for the BBS in Parkinson’s disease has been reported to be .94, indicating a 94% probability that repeated administrations of the test would yield the same result (Steffen & Seney, 2008).
Interrater reliability refers to the consistency in test results when different testers administer the assessment tool. For instance, the interrater reliability for the BBS for clients with multiple sclerosis was reported to be .96 (Cattaneo, Jonsdottir, & Repetti, 2007).
Internal consistency refers to the degree to which different items within an assessment tool measure the underlying construct. Values for internal consistency range from 0 to 1, with higher values indicating that items within an assessment measure the same construct. For example, the internal consistency of the BBS for clients with Parkinson’s disease was found to be .86, indicating that this assessment is good for measuring balance (Steffen & Seney, 2008).
Potential Changes in Medicare Reimbursement Policy
The Centers for Medicare and Medicaid Services (CMS) recently announced a potential change in reimbursement policy (Center for Medicare Advocacy, 2012; see www.medicareadvocacy.org/hidden/highlight-improvement-standard/). At present, reimbursement for occupational therapy services is based on the improvement standard: Practitioners must demonstrate improvement in clients to be reimbursed for services provided. The improvement standard has meant that people with progressive disorders are often denied therapy services if they did not improve over the course of therapy (typically 6–8 wk).
The potential change in Medicare policy comes on the heels of a nationwide class-action settlement, approved in January 2013, for provision of skilled health services (“A Humane Medicare Rule Change,” 2012). Under the settlement, reimbursement for therapeutic services will be based on a need standard; Medicare will cover services if they are needed to maintain the client’s condition or to prevent or slow the decline in function. According to CMS, the new settlement applies to Medicare Parts A and B and specifically covers occupational, physical, and speech therapy in outpatient, home health, and skilled nursing facilities. The change in policy is significant for occupational therapy practice because it may provide reimbursement for a large number of clients who are currently denied coverage. In light of these policy changes, occupational therapy practitioners must select appropriate assessment tools to accurately document the need for services to maintain and improve function in clients with chronic progressive disorders.