Abstract
The Cognitive Performance Test (CPT) is a standardized occupational therapy assessment that examines cognitive integration with functioning in an instrumental activities of daily living context. Conventional cognitive measures provide diagnostic utility but do not fully address the functional implications. Ninety-one veterans diagnosed with cognitive impairment were evaluated. We compared the predictive value of the CPT with the Large Allen Cognitive Level Screen (LACLS), Mini-Mental State Examination (MMSE), and Montreal Cognitive Assessment (MoCA) for the need to retire from driving versus ability to pass an on-road exam. Measures were also analyzed by diagnostic classification. CPT correctly classified a mild versus major neurocognitive disorder, whereas MMSE, MoCA, and LACLS did not differentiate the groups. A CPT cutoff score of <4.7/5.6 showed 89% sensitivity for failing the road exam and 75% specificity for ability to pass. CPT discriminated functional level in neurocognitive disorders and had better predictive value for fitness to drive compared with conventional cognitive measures.
There is a growing interest in performance-based assessments that offer both clinical utility and ecological validity for predicting levels of competency required in everyday tasks (Burgess et al., 2006; Dickerson, Reistetter, Davis, & Monahan, 2011; Hartman-Maeir, Katz, & Baum, 2009; Moore, Palmer, Patterson, & Jeste, 2007; Schaber et al., 2013). Ecological validity refers to whether the findings in a controlled environment (i.e., the clinic) can be generalized to the real world. The goal of occupational therapy evaluation is predictive validity for real-world functioning with an emphasis on the ability to measure change in functioning over time. Functional cognition is a relatively new concept that links the two constructs of function and cognition in the performance of daily life. Older adults often show a decline in their performance of instrumental activities of daily living (IADLs), and impairment in IADLs can predict the onset of future dementia (Pérès et al., 2008; Sikkes et al., 2011).
The fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2013) subsumes dementia under the new category of neurocognitive disorders (NCDs). Neurocognitive disorders are further divided into major and mild NCDs, according to the degree of functional impairment. Dementia is classified as a major NCD with the diagnostic criteria that cognitive deficits interfere with independence in everyday activities and, at a minimum, that assistance is required with IADLs. The diagnostic criterion for a mild NCD is that deficits do not interfere with IADL independence, although greater effort, compensatory strategies, and accommodation may be required.
Diagnostic criteria for etiological subtypes of NCDs (e.g., Alzheimer’s disease [AD], vascular, frontotemporal, or Lewy body disease) are based on the neuropathology of the disease process and pattern of cognitive domains affected. Thus, the neurocognitive profiles of NCDs are known to be heterogeneous. Memory is typically the first domain affected in AD, but not in all NCDs. Complex attention, executive function, language, perceptual–motor processing, and social cognition are among other domains affected. Although the neurocognitive patterns differ across etiological subtypes, the pattern can also differ in the same NCD, such as an atypical AD pattern that presents initially with greater language and executive dysfunction than memory dysfunction (American Psychiatric Association, 2013).
Dementia is diagnosed clinically on the basis of analysis of both cognitive and functional indicators. Neuropsychological test batteries use a bottom-up approach to analyzing brain function by gathering information about impairments in specific cognitive domains. Although this micro-level assessment process gives an accurate depiction of specific cognitive impairments, the relationship between components of cognition and overall function is unclear. Cognitive screening tools are also used in diagnosis, the most prevalent being the Mini-Mental State Examination (MMSE; Folstein, Folstein, & McHugh, 1975). The more recent Montreal Cognitive Assessment (MoCA; Nasreddine et al., 2005) was developed to provide better sensitivity for distinguishing between mild cognitive impairment (MCI) and normal cognitive function. Although cognitive screens provide diagnostic utility and an indication of severity level (no impairment, mild impairment, significant impairment), they do not fully address the functional implications, in particular those associated with the wide range of significant impairment in mild- to moderate-stage disease.
The Cognitive Performance Test (CPT; Burns, 2018) is a standardized performance-based assessment used to explain and predict the client’s capacity to function in various contexts and guide intervention plans. It measures and tracks the severity of a cognitive-functional disability by means of baseline and serial assessment (Burns, McCarten, Adler, Bauer, & Kuskowski, 2004; Burns, Mortimer, & Merchak, 1994; Douglas, Letts, Richardson, & Eva, 2013). The CPT uses a top-down approach to analyzing function, relying on observation of performance on everyday tasks to ascertain cognitive abilities. It examines cognitive integration with functioning in an IADL environmental context, and in particular it rates executive control function, the group of cognitive processes that mediate goal-directed activity (Levy & Burns, 2011).
Reliability and validity were established as part of a longitudinal study of AD conducted by the National Institute on Aging (Burns et al., 1994). Seventy-seven community-residing adults with mild to moderate AD were evaluated annually over 4 yr. The CPT was found to have good psychometric properties and is referenced as a validated ecological approach to assessment in AD (Beck & Frank, 1997; Douglas, Letts, Eva, & Richardson, 2012; Hartman-Maeir et al., 2009; Moore et al., 2007). Psychometric properties include test–retest reliability (r = .91), interrater reliability (r = .89), internal consistency of subtasks (r = .84), and concurrent validity with the MMSE (r = .67) and the Lawton IADL scale (r = .64) and ADL physical self-maintenance scale (r = .49; Lawton & Brody, 1969). A significant difference was found between CPT scores of AD participants versus control participants, with no control participants scoring below 5.2 on the CPT (range = 1.0 [late-stage dementia] to 5.6 [intact functioning]). Longitudinal testing (N = 64) demonstrated a significant decline in mean CPT scores with disease progression, and in contrast to the MMSE, initial CPT scores predicted the risk of institutionalization over the 4-yr follow-up period. All participants with an initial CPT score of 4.2 or less were institutionalized within 3.6 yr of the baseline assessment, whereas 40% of participants whose initial scores were above 4.2 remained in the community. Reliability and validity have also been demonstrated by Bar-Yosef, Weinblatt, and Katz (2000) in studies of AD and control participants in Israel, and by Douglas et al. (2012) in studies of hospitalized older adults.
In the Occupational Therapy Practice Framework: Domain and Process (3rd ed.; American Occupational Therapy Association, 2014), community mobility and driving are identified as IADLs in the domain of practice. Driving requires the integration of complex cognitive resources, including complex attention, planning, and multitasking or the ability to divide attention to environmental cues: These are the same cognitive performance skills assessed by the CPT. At the Minneapolis Veterans Affairs Medical Center Drivers Rehabilitation Clinic, an occupational therapist driving rehabilitation specialist (DRS) uses the CPT as the predriving screen, or clinical portion of the driving evaluation, instead of more traditionally used tools, such as the MMSE, MoCA, or Large Allen Cognitive Level Screen (LACLS; Allen, 1990; Kehrberg, Kuskowski, Mortimer, & Shoberg, 1992). Cognitive screening tools target the essential visual and cognitive functions for driving and are important for triggering the need for further evaluation. However, further research is needed to determine their validity in real-world application (Dickerson et al., 2011). We compared the predictive value of the CPT with these cognitive screen measures for the need to retire from driving versus the ability to pass an on-road exam. We also analyzed these clinical measures to see how scores classified clients by a diagnosis of mild versus major NCD.
Method
Retrospective data were studied from veterans (N = 91) who had been evaluated for cognitive impairment either in the Geriatric Research, Education and Clinical Center (GRECC) or in primary care through our Dementia Demonstration Project (DDP; McCarten et al., 2012), and were subsequently referred for the CPT and on-road exam. The GRECC has institutional review board approval to use deidentified clinical data for research purposes. The participants’ mean age was 78.8 (standard deviation [SD] = 7.7 yr; range = 59–91). All underwent a driving evaluation that included tests of vision, the CPT, and the on-road exam. No difference was found in mean age between participants who passed and those who failed the road exam. Fifty-five participants had MMSE scores ranging from 6 to 29 with a mean of 22 out of 30 (SD = 5.3); the other 36 participants were from the DDP, where the MoCA was used, with scores ranging from 12 to 26 with a mean of 19 out of 30 (SD = 3.5). Fifty-seven participants had LACLS scores ranging from 3.8 to 5.4 with a mean of 4.7 out of 5.8 (SD = 4.1). The LACLS was administered as a screen for cognitive impairment at the time of or within 2–4 wk of the driving evaluation.
Instruments
Cognitive Performance Test.
The CPT consists of seven subtasks, for which the task cues and working memory requirements are systematically varied to assess ordinal levels of functional cognition. Four subtasks (Medbox, Shop, Phone, and Travel) scale to Level 6, and three (Wash, Toast, and Dress) scale to Level 5 because these involve less complex processing requirements. With a ceiling of 5.6, the total score is used in practice and represents an average of the subtask scores. All subtasks are a repeated measure of the same construct, the client’s level of performance in working memory and executive control functions (e.g., task planning, problem solving, divided attention, new learning). At each higher CPT level, the subtask cues used in performance are more complex, resulting in working memory behavior that is more organized and complex. CPT scores are interpreted within its half-level profile system, which delineates the predicted function and needs around scores at the half-level (Table 1). CPT profiles have empirical support for IADL and activity of daily living (ADL) performance capacities and assistance needs (Bar-Yosef et al., 2000; Burns et al., 1994, 2004; Douglas et al., 2012, 2013; Grabe, Schroll, Vig, & Schaber, 2009; Levy & Burns, 2011; Schaber et al., 2013; Thralow & Rueter, 1993) and are meant to be individualized and mediated by client factors; these could include level of self-awareness, physical disability, cultural factors, and idiosyncratic factors related to skills in particular daily activities.
CPT Cognitive-Functional Profiles
Note. ADLs = activities of daily living; CPT = Cognitive Performance Test; IADLs = instrumental activities of daily living. From Cognitive Performance Test Revised Manual (p. 4), by T. Burns, 2018, Pequannock, NJ: Maddak. Copyright © 2018 by Theressa Burns. Reprinted with permission.
The full battery was administered to all 91 participants, although the CPT5, a shorter version developed for primary care, was often used in the DDP. The CPT5 includes Medbox, Shop, Toast, Phone, and Wash, and we analyzed this commonly used battery for comparison. The CPT seven-task version has been compared with the five-task version for concurrent validity using an Altman–Bland comparison method (Ewald, 2009). The bias (−0.01; potential for the score between the two versions to increase or decrease) and limits of agreement (−0.30 to 0.27) indicate that the five-task version can be used effectively in lieu of the seven-task version although, with fewer subtasks to score and average, the CPT5 typically yields a total score a few decimal points lower. The predictive value of the CPT Toast subtask, a single task design used in the original version of the Savvy Caregiver Program (Hepburn, Lewis, Sherman, & Tornatore, 2003), was also analyzed for functional level.
Large Allen Cognitive Level Screen.
The LACLS is a screening tool designed to assess the cognitive level of people with psychiatric illness or cognitive impairment (Allen, 1985, 1990). It requires the person to replicate three increasingly complex leather lacing stitches and yields a decimal score on a six-level scale (6 = no impairment) based on error frequency and type, ability to recognize and correct errors, and the level of assistance required. We continued using the 1990 scoring criteria that include both odd and even decimal-mode scores. The LACLS was developed for the older adult population and was validated against the smaller tool (the Allen Cognitive Level Screen; Allen, 1990) in people with dementia (Kehrberg et al., 1992). Performance on the Allen Cognitive Level Screen has been related to ability to carry out basic ADLs (Keller & Hayes, 1998), community functioning (Su, Tsai, Su, Tang, & Tsai, 2011), and elements of cognition (Mayer, 1988); a cutoff score of 5.2 is suggested for retirement from driving (Allen, Blue, & Earhart, 1995).
Mini-Mental State Examination and Montreal Cognitive Assessment.
Both the MMSE and the MoCA are interdisciplinary cognitive screening tools scored on a 30-point scale. A cutoff of <24 is used to indicate impairment on the MMSE; the MoCA uses <26. The MoCA’s memory testing involves more words, fewer learning trials, and a longer delay before recall. Executive functions, higher level language abilities, and complex visual–spatial processing are also assessed, presenting more numerous and demanding tasks than the MMSE (e.g., drawing a clock in MoCA vs. intersecting pentagons in MMSE). Thus, the MoCA assesses executive functioning and abstract thought (e.g., describing the similarity between a watch and ruler), whereas the MMSE does not tap into these areas.
Both the MMSE and MoCA have been studied in relation to driving. The MMSE has been shown to relate to road performance in terms of score severity in some studies but not others (Molnar, Patel, Marshall, Man-Song-Hing, & Wilson, 2006). Likewise, the MoCA has yielded similar findings. Esser et al. (2016), for example, reported that a MoCA score between 12 and 26 out of 30 points provided only a 50% chance of passing versus failing the road exam.
On-Road Exam.
The on-road exam was completed in stages by the DRS and discontinued with poor performance at any stage. After an orientation to the standard vehicle with brake control for the DRS, participants were asked to demonstrate driving ability in the parking lot and progressed to more complex driving environments, including residential streets, highways, and freeway driving. For participants who scored in CPT Profile 4.5, where new learning is difficult, a local driving evaluation may have been done to assess the participant’s ability to drive to specific locations (i.e., grocery store, coffee shop, bank, church, or barber).
Procedure
We compared the predictive value of the CPT, LACLS, MMSE, and MoCA for the need to retire from driving versus ability to pass the on-road exam. Measures were also analyzed to see how scores classified participants by consensus diagnosis, a process in which the differential diagnosis is agreed upon by physicians and neuropsychologists who review the person’s physical exam, history of cognitive and functional symptoms, brain imaging studies, neuropsychological cognitive domain measures, and CPT profile. The differential diagnosis of neurocognitive disorders is a clinical determination based on the person’s presenting symptoms.
We investigated the ability of the above measures to correctly classify functional decline in participants with AD or another type of dementia versus MCI or other mild NCD. Participants fell into the four diagnostic groups listed in Table 2: cognitive impairment–not demented (CIND; n = 9), MCI (n = 13), AD (n = 52), and non-AD dementia (n = 17). In our center, the differential diagnosis of MCI is given for a mild NCD that is likely to progress to AD. The four diagnostic groups were compared on the dependent variables, including the CPT, CPT5, CPT–Toast, LACLS, MMSE, and MoCA. Analysis of variance, followed by Bonferroni-adjusted pairwise group comparisons, was used. Sensitivity, specificity, and positive predictive value (the percentage of participants who failed a specified test who were told to retire from driving) and negative predictive value (the percentage of participants who passed a specified test who were capable of passing the on-road exam) were calculated for the dependent measures as they related to the outcome of driving retirement versus driving restriction for various prespecified cutoff scores determined by clinical observation of concern.
Consensus Diagnoses by Clinical Measure With Confidence Intervals
Note. AD = Alzheimer’s disease; CI = confidence interval; CIND = cognitive impairment–not demented; CPT = Cognitive Performance Test; CPT5 = five-item version of the CPT; CPT–Toast = Toast subtask of the CPT; LACLS = Large Allen Cognitive Level Screen; M = mean; MCI = mild cognitive impairment; MMSE = Mini-Mental State Examination; MoCA = Montreal Cognitive Assessment; SD = standard deviation.
p < .001 vs. AD and p = .003 vs. dementia groups.
p < .001 vs. AD and p = .003 vs. dementia groups.
p < .001 vs. MCI and CIND groups.
Results
Group Comparisons
An analysis of variance by diagnosis revealed that the four groups differed on the CPT (p < .001), CPT5 (p < .001), and MoCA (p = .048). Groups were not significantly different on the MMSE (p = .062), LACLS (p = .123), or CPT–Toast (p = .114). For the CPT, post hoc comparisons revealed that the CIND group was significantly different from the AD group (p < .001; all p values are Bonferroni corrected) and from the dementia group (p = .003), but not from the MCI group (p = .765). The AD group was significantly different from the MCI group (p < .001) but not from the dementia group (p = .669). Finally, the MCI group was significantly different from the dementia group (p = .003). The pattern of group differences was identical for the CPT5. None of the two-group post hoc comparisons for the MoCA had p < .05 after Bonferroni correction for multiple comparisons. In the original studies of the CPT, a score of 5.1 was the cutoff for persons with no impairment versus those with AD (Burns et al., 1994). These findings were replicated in our study, with all dementia diagnostic groups scoring at or below 5.1 on the CPT.
Physicians seldom refer clients in CPT Profile 5 (i.e., MCI) for the on-road exam and most often refer clients who score in CPT Profile 4 (mild to moderate stage). However, in this study 42 of 91 participants did represent clients in Profile 5, with scores ranging from 4.9 to 5.3. Eighteen of 42 participants scored at 4.9, the point at which MCI often overlaps with early AD. Therefore, most participants already had a diagnosis of major NCD, and there were often concerns about driving, leading to the DRS referral. In our practice, the goal is to restrict driving to local, low traffic and speed to reduce the safety risks, given adequate performance abilities on the road. The overall goal is to assist the person in the eventual need to retire from driving, with 6-mo follow-up visits.
Sensitivity and Specificity
Table 3 lists the predictive value of the various measures for driving. With a cutoff below 4.7, the CPT correctly identified participants who needed to retire from driving 89% of the time (sensitivity) and correctly identified those who could drive with restrictions 75% of the time (specificity). For the CPT5, the 4.7 cutoff showed 92% sensitivity for need to retire with lower specificity at 63%. For the CPT–Toast subtask, whole numbers are scored: With a Savvy Caregiver Program cutoff at 4 or below 5, sensitivity was poor, at 23%, with 100% specificity that participants who passed the road exam scored above 3.
Sensitivity and Specificity for Clinical Measures
Note. CPT = Cognitive Performance Test; CPT5 = five-item version of the CPT; CPT–Toast = Toast subtask of the CPT; LACLS = Large Allen Cognitive Level Screen; M = mean; MMSE = Mini-Mental State Examination; MoCA = Montreal Cognitive Assessment; SD = standard deviation.
Percentage of participants below the cutoff score who were asked to retire from driving.
Percentage of participants above the cutoff score who could pass the on-road test.
Two LACLS cutoff scores were analyzed. Allen et al. (1995) used a modal interpretive system that identifies a cutoff of 5.2 for retirement from driving. In our study, a cutoff of 5.2 on the LACLS misclassified participants the majority of the time. A score of 4.7 had better predictive value, although 40% of those who scored below 4.7 were still able to pass the road exam. The LACLS did not differentiate a mild versus major NCD, with only a trend that higher scores in Level 4 were associated with a mild NCD and lower scores with a major NCD. Similar findings regarding functional level were found in the MMSE and MoCA, which did not differentiate between mild and major NCD. The CPT5 demonstrated an ability to differentiate the two groups, but with less specificity than the full CPT for passing the road exam. The CPT–Toast subtask predicted neither driving capacity nor the correct level of function.
A recent study conducted by Wesson et al. (2017) of the revised tool, the LACLS–5 (Allen et al., 2007), showed the ability to discriminate between a mild versus major NCD. However, the reported cut score of 4.6 and above for a mild NCD falls in the range of clear dementia and associated functional impairment on the CPT (Table 1). This demonstrates the problem with reliability when using screening measures to interpret daily function. Nevertheless, both the LACLS (Allen, 1990) and LACLS–5 demonstrate clinical utility as valid cognitive screening tools and, in particular, offer a cognitive-functional approach to screening for occupational therapy practitioners.
Discussion
Some clients with early AD continue to drive safely, and beginning to transition a client away from driving may be more appropriate than immediate driving cessation (Ott et al., 2008). With good caregiver supports, the DRS can assist with a plan to increase safety while limiting driving time and distance, planning for alternative forms of transportation, and helping the client understand and accept the idea of eventually discontinuing driving. Although disease duration is associated with driving impairment, the rate of progression is not uniform across all people. Therefore, measures of disease severity instead of disease duration are more useful as predictors of impaired driving (Carr, Barco, Wallendorf, Snellgrove, & Ott, 2011; Iverson et al., 2010). The CPT provides a severity measure that stages functional abilities required for safe driving. The results show that the CPT is a stronger measure of driving performance than the cognitive screens used in clinical practice at our medical center. This study supports the position posed by Dickerson et al. (2011) that experienced occupational therapists in general practice can address driving through an IADL performance-based assessment and should be able to determine who is a safe driver, who is at risk for unsafe driving, and who needs further evaluation by a specialist.
The Cochrane Dementia and Cognitive Improvement Group reported that modern theories of driver behavior have yet to be incorporated into existing driver assessment procedures, and these almost invariably emphasize cognitive measures, which may play a less determining role in driving efficacy than is commonly understood (Martin, Marottoli, & O’Neill, 2013). The CPT, consisting of familiar daily tasks, may have more relevance when performance errors show a decline in previous abilities or difficulty with a change in environment (i.e., the clinic) in a common task done successfully from procedural memory in one’s own home. Family members observe the CPT assessment and participate in discussion and plans about capacity and needs in all relevant IADLs. Of the 35 veterans evaluated at a 6-mo follow-up, 78% had complied with the DRS recommendations to either retire from or restrict their driving.
Driving is the most common form of transportation for older people, and access is an important factor in maintaining social inclusion and participation. Considerable evidence indicates that limiting this access (i.e., by driving cessation) is associated with poor health and depression (Marottoli et al., 2000) and a higher risk of nursing home placement (Freeman, Gange, Muñoz, & West, 2006). Interventions that will maintain safe mobility for the longest period for persons with dementia are unknown. An assessment for dementia allows for advance planning of driving withdrawal and substitution of other modes of transportation. It also allows for regular retesting to guide this process. The importance of transportation to health and social inclusion of older people and, in particular, older people with dementia must be recognized (Martin et al., 2013).
Limitations
Our research has several limitations. Three occupational therapists administered the CPT, including two DRSs who evaluated driving performance on the road. However, all raters were skilled in administration of the CPT, and the CPT has strong interrater reliability (Bar-Yosef et al., 2000; Burns et al., 1994; Douglas et al., 2012; Schaber, Stallings, Brogan, & Ali, 2016). The decision to pass, fail, or recommend restrictions to driving is based on the clinical judgment of the DRS. Unfortunately, the on-road exam cannot be completely standardized because of the nature of a community drive. However, routes incorporated progression of increased cognitive demand, typical for the DRS on-road assessment process. Although the MoCA showed good sensitivity and specificity for driving, the small sample size (N = 36) limits interpretation and generalization.
Implications for Occupational Therapy Practice
Occupational therapy practitioners evaluate clients for cognitive-functional impairment in many practice settings and are skilled in observing the effects of such impairment in the context of complex daily tasks. Early identification of dementia is recognized as a proactive approach to addressing the trajectory of functional decline and prevention of the typical problems associated without recognition. The CPT provides a standardized clinical assessment process for determining cognitive-functional levels and evidence-based profiles for intervention. This study has the following implications for occupational therapy practice:
Practitioners assist in the diagnostic workup of NCDs by providing a functional diagnosis on the CPT.
Practitioners can measure changes in the cognitive-functional level with serial assessment.
Family observe the CPT assessment and actively participate in education and care planning.
CPT profiles are evidence based for clinical intervention in driving and all IADLs and ADLs and are mediated by client factors.
A CPT cutoff score of <4.7 can be used in discussion about the need to either restrict or retire from driving, in particular when there is evidence of driving concerns or a client’s score is already in low Profile 4.
Occupational therapy programs need to be used in an effective and efficient manner with an appropriate referral system that meets clients’ needs.
Footnotes
Acknowledgments
Theressa Burns is the author of the Cognitive Performance Test and has a financial relationship with Maddak Inc., which produces the kit of standard props. We thank the occupational therapists who assisted in reviewing and preparing this article for submission: Kristine Haertl of Saint Catherine University; Stacy Smallfield of Washington University in Saint Louis; and Deb Voydetich, Veterans Affairs Liaison for the Central Office.
