Abstract
This study provides evidence for establishing functional cognition as a unique construct that is distinct from executive function as well as from fluid and crystallized cognition, and supports occupational therapy professionals in establishing the profession’s role in evaluating and treating functional cognition deficits to support patients’ return to desired occupations in the family, workplace, and community.
The ability to manage the complexities of daily life involves setting goals, identifying and overcoming problems, and taking actions to perform the tasks and activities of choice. This process of management entails a complex interplay among biological, psychological, and social influences and a satisfactory match among the person, the task, and contextual characteristics. At the confluence of biopsychosocial factors and the person–task–context fit is occupational performance, the term that occupational therapists use to describe an individual interacting with the environment while performing activities that are considered important and meaningful (Baum & Edwards, 1995; Baum & Law, 1997).
Many people with chronic disorders or disabilities of neurological, mental, or metabolic processes have cognitive problems that limit their occupational performance. The performance of goal-directed activities requires the effective use of cognitive processes to plan complex activities, anticipate potential errors, self-correct, make decisions, use judgment, dynamically respond to changes in the environment, and make wise choices as the individual navigates the challenges and difficulties of life (Lezak et al., 2004; Toglia, 1991). Thus, impairment or loss of these processes compromises the ability to achieve goal-oriented performance and thereby limits the performance of desired activities and participation in one’s roles and society.
To assist people with even subtle cognitive impairments in achieving their required and desired occupations, clinicians use a client-centered strategy that considers: what the person needs and wants to do; the cognitive and environmental demands of those activities; the client’s perception of their limitations; and the client’s capacity for everyday performance (Hartman-Maeir, Katz & Baum, 2009). The practitioner explores with the client what he or she perceives to be the issues that limit their participation and cause difficulty in carrying out tasks, including those related to productivity and work, personal care, home maintenance, sleep, and recreation or leisure. In addition, the practitioner needs to evaluate the context in which the activity will be performed and whether that context supports performance, considering whether environmental modifications can facilitate person–environment fit. Finally, determining an individual’s capacity for real-world or everyday performance is essential (Alderman et al., 2003; Fisher, 1998; Shallice & Burgess, 1991). Capacity is determined by having the person demonstrate that he or she can actually perform the activity with or without support, adaptations, or modifications.
Performance-based testing originated from environmental psychology theory and the concept of ecological validity proposed by Brunswik (1955). He advocated for research that centers the observation of behavior in an unrestricted rather than in a controlled laboratory environment. He referred to such testing as being “ecologically valid,” meaning representative of real-world performance requiring multitasking that occurs in environments that may not be supportive. Ecologically valid assessments contrast with paper-and-pencil neuropsychological assessments, which gather the results of tests of component cognitive skills in controlled environments and are used to extrapolate these results to the performance of daily life tasks. Shallice and Burgess’s (1991) introduction of a multitasking assessment, the Multiple Errands Test (MET), was of particular interest to occupational therapists, because it closely matches the goals of occupational therapy to help clients perform the complex activities of their daily lives.
Since Shallice and Burgess (1991) introduced ecologically valid performance-based assessments, occupational therapists have both incorporated the MET into their practice on the basis of scientific evidence (Dawson et al., 2009; Morrison et al., 2013) and developed other performance-based measures to assess the interaction of the person performing an activity in an environment (Árnadóttir & Fisher, 2008; Baum et al., 2008; Baum & Edwards, 1993; Fisher, 1995; Hartman-Maeir, Harel, & Katz, 2009; Holm & Rogers, 2008). Results from performance assessments enable therapists to help their clients as they face the challenges of setting goals and planning and implementing tasks that are necessary and important as they return to their daily lives. Through the assessment of cognitive capability in the performance of daily tasks, this approach determines the person’s capacity to be safe, live alone, work, or perform any task that is important and meaningful to them. Key cognitive processes are observed in the performance of daily life activities (Baum et al., 2008; Baum & Edwards, 1993), including task initiation, the process that precedes the performance of a task (Lezak et al., 2004); organization, the physical arrangement of the environment, tools, and materials to facilitate efficient and effective performance (Baum et al., 2008; Weld & Evans, 1990); judgment (Goel et al., 1997; Lezak et al., 2004); and completion (Goel et al., 1997).
The occupational therapist approaches the measurement of cognition and function not just to know what a person can do but also to understand how to foster the individual’s engagement in daily life. Occupational therapists have developed many instruments to assess the cognitive issues that support or limit client performance. The construct underlying these tools is functional cognition. However, none of these tools have been validated as tapping a construct of functional cognition that is distinct from fluid and crystallized cognitive abilities. Functional cognition may be thought of as the cognitive processes that support performance, or the “doing,” of complex everyday activities.
A crucial issue that has not been resolved is whether functional cognition is distinct from other well-established cognitive constructs; namely, fluid and crystallized cognitive abilities as assessed by means of neuropsychological tests (e.g., Horn & Cattell, 1966). In prior work on the development of performance-based assessments, the correlations among the neuropsychological assessments of fluid abilities with those of functional cognitive skills have been modest at best. For instance, Baum et al. (2008) described the utility of the Executive Function Performance Test (EFPT) in distinguishing older adults with no, mild, or moderate stroke impairment. In addition to the EFPT, the participants completed several neuropsychological assessments that were deemed to tap fluid cognitive abilities, such as the Short Blessed Test (Katzman et al., 1983), Trail Making Test (Reitan & Wolfson, 1995), and the Digits Backward subtest of the Wechsler Memory Scale–Revised (Wechsler, 1987). Although there were significant differences in performance on the EFPT between participants without stroke and those with mild and moderate stroke on the performance-based and neuropsychological assessments, correlations between EFPT total scores and the scores on tests of fluid cognition were only modest in magnitude (rs = .39–.59; Baum et al., 2008). In fact, the correlation between the EFPT and the Functional Assessment Measure (FAM; Hawley et al., 1999)—a tool that assesses the performance of instrumental activities of daily living—was higher, at .68 (Baum et al., 2008). These results raised the question of whether the difference in magnitude of the correlation between the EFPT and the FAM, and between the EFPT and neuropsychological assessments of fluid cognition, is due to differences in the underlying cognitive constructs or to other task demands.
The goal of this investigation was to determine whether functional cognition is distinct from crystallized and fluid cognitive abilities, which are typically evaluated with neuropsychological assessments. The EFPT was included in a large study, “Measurement of Medical Rehabilitation Outcomes” (principal investigator: A. W. Heinemann), along with neuropsychological assessments and the NIH Toolbox Cognition Battery, offering an opportunity to determine whether performance-based tests are appropriate assessments of fluid or crystallized cognitive constructs or whether functional cognition is a unique construct.
Method
Participants
Participants with spinal cord injury, traumatic brain injury, or stroke were recruited from three facilities: the Rehabilitation Institute of Chicago (now the Shirley Ryan AbilityLab), Washington University in St. Louis, and the University of Michigan in Ann Arbor. Participants were living in the community, able to come to a testing site, and at least 18 yr old; were at least 1 yr postinjury and able to comprehend and speak English at a fifth-grade reading level; had adequate near vision; did not have aphasia; were able to provide informed consent; and were willing and able to return for follow-up testing. Data were collected in accordance with, and with the approval of, the local institutional review boards.
Instruments
NIH Toolbox Cognition Battery
The NIH Toolbox Cognition Battery (NIHTB–CB; Gershon et al., 2010) was developed under the commission of the National Institutes of Health to offer brief and efficient measures for use in clinical research. The NIHTB–CB (Weintraub et al., 2013) is a computerized assessment consisting of seven performance-based tests that measure fluid and crystallized cognition. Crystallized cognition is assessed with two tests; namely, the Picture Vocabulary Test (receptive vocabulary; Gershon et al., 2013, 2014) and the Oral Reading Recognition Test (verbal knowledge; Gershon et al., 2013, 2014). Fluid cognition is assessed with five tests, including the Picture Sequence Memory Test (for episodic memory; Bauer et al., 2013; Dikmen et al., 2014), the Pattern Comparison Processing Speed Test (for processing speed; Carlozzi et al., 2013, 2014), the List Sorting Working Memory Test (for working memory; Tulsky et al., 2014, 2017;), the Flanker Inhibitory Control and Attention Test (for inhibitory control and attention; Zelazo et al., 2013, 2014), and the Dimensional Change Card Sort Test (for cognitive flexibility; Zelazo et al., 2013, 2014). Detailed descriptions of each test are presented in the Supplemental Appendix (see Table A.1, available online with this article at https://research.aota.org/ajot) and have also been reported elsewhere (Carlozzi et al., 2017). Raw scores from each test and composite scores of fluid and crystallized cognition were converted to normally distributed standard scores with a mean of 100 (SD = 15), where higher scores reflect higher levels of cognitive functioning. The validity of the NIHTB–CB has been examined extensively and established in people with neurological conditions (Carlozzi et al., 2017).
Executive Function Performance Test
The Executive Function Performance Test (EFPT; Baum et al., 2008) assesses the level of support needed to perform four instrumental tasks: simple cooking, using a telephone, managing medications, and paying bills. For each task, five components are assessed: initiation of a task (beginning the task), organization (retrieval and arrangement of tools and supplies), sequencing (execution of steps in a correct order), safety and judgment (avoiding a dangerous situation), and completion (deciding and acknowledging that a task is complete). Detailed descriptions of each component are presented in Table 1. Administrators can provide five levels of cueing: 0 (no cue required), 1 (verbal guidance), 2 (gestural guidance), 3 (direct verbal assistance), 4 (physical assistance), and 5 (do for the participant). The highest level of cueing necessary to support task performance is recorded; thus, higher scores reflect more severe deficits. Three scores are computed: the component score, the task score, and a total score. The component score is calculated by summing the numbers recorded on each of the four tasks for initiation, organization, sequencing, safety and judgment, and completion. Scores on each component range from 0 to 5, and the total for all four tasks ranges from 0 to 20. The task score is calculated by summing the five scores for each task. The range for each task is 0 to 25. The total score is the sum of the performance on all four tasks; the total score of performance on all four tasks ranges from 0 to 100. People with motor impairment are not penalized if they ask for help because the impairment necessitates physical assistance. The reliability and validity of the EFPT have been established in neurological populations, including people with stroke (Baum et al., 2008), traumatic brain injury (Baum et al., 2017), and multiple sclerosis (Goverover et al., 2005).
Sample Characteristics
Results are from independent t tests and χ2 tests.
Neuropsychological Tests
We also administered the gold standard battery of neuropsychological tests to assess various domains of cognition, including receptive vocabulary, as measured by the Peabody Picture Vocabulary Test–Fourth Edition (Dunn & Dunn, 2007); reading, as measured by the Wide Range Assessment Test–Fourth Edition (Wilkinson & Robertson, 2006); episodic memory, as measured by the Brief Visuospatial Memory Test–Revised (Benedict et al., 1996); working memory, as measured by the Paced Auditory Serial Addition Test (Gronwall, 1977) and the Letter–Number Sequencing subtest of the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS–IV; Wechsler, 2008); processing speed, as measured by the WAIS–IV Digit–Symbol and Symbol Search subtests (Wechsler, 2008); and executive function, as measured by the 64-card version of the Wisconsin Card Sorting Test (Kongs et al., 2000) and the Delis–Kaplan Executive Function System Color–Word Interference Test (Delis et al., 2001).
NIH Toolbox Motor Battery
The NIH Toolbox Motor Battery (NIHTB–MB) assesses five domains of motor functioning: dexterity (with the 9-Hole Pegboard Dexterity Test), strength (with the Grip Strength Dynamometer), balance (with the Balance Accelerometer Measure), locomotion (with the 4-Meter Walk Test), and endurance (with the 2-Minute Walk Test; Carlozzi et al., 2017). The NIHTB–MB has been validated previously in people with stroke (Carlozzi et al., 2017).
Statistical Analyses
To investigate the factor structure of cognition, we performed exploratory factor analyses (EFAs) followed by confirmatory factor analyses (CFAs). We randomly split our sample into equal halves, with one half included in the EFA and the other in the CFA. First, we explored the number of factors underlying cognition by means of EFA using the oblique Geomin rotation. We determined the number of factors on the basis of scree plots and the eigenvalues-greater-than-1 rule (Nunnally, 1978). We then identified the factor on which the items primarily loaded using a cutoff of .40 (Heinemann et al., 2013). Second, on the basis of the EFA results, we examined and compared competitive models (first-order, second-order, and bifactor models) by means of CFA using the robust maximum likelihood estimation procedure. In both the EFA and CFA, we evaluated the goodness-of-fit of each model and selected the final model on the basis of several fit indices. We used the comparative fit index (CFI) and Tucker–Lewis Index (TLI) as indicators of incremental model improvement; values of .95 or higher indicate good model fit (Browne & Cudeck, 1993). The root-mean-square error of approximation (RMSEA) represents the closeness of fit; values of .08 or less indicate a good fit (Hu & Bentler, 1999). The Akaike information criterion (AIC; Akaike, 1974) was computed to aid model comparison; differences in AICs less than 10 indicate that the model with a smaller AIC has a significantly better fit. Among two models with comparable fit, we selected the simpler model on the basis of the parsimony principle. To examine the associations between the identified factors of cognition and different subdomains of cognition, we examined the Pearson correlation coefficients (rs) between each identified factor of cognition with different neuropsychological tests. The greater the absolute value of r, the stronger the correlation. Thresholds for weak, moderate, and strong correlations were considered to be .3, .5, and .7, respectively (Hinkle et al., 2003). All analyses were performed using IBM SPSS Statistics (Version 23) and Mplus (Muthén & Muthén, 2015).
Follow-Up Analyses
To identify whether the establishment of functional cognition as an independent construct in the EFA was due to the motor demands of the EFPT, we conducted a maximum likelihood factor analysis with varimax rotation for the EFPT and NIHTB–MB.
Results
Sample Characteristics
Table 1 shows the sociodemographic and clinical characteristics of the sample, the split sample for EFA, and the split sample for CFA. No significant differences in sociodemographic and clinical characteristics were found between the EFA and CFA samples. Our total sample included 493 participants (146 with traumatic brain injury, 180 with stroke, and 167 with spinal cord injury). The mean age and yr since injury of the overall sample were 48.22 and 6.96 yr, respectively. Participants were predominantly men (63.1%), non-Hispanic White (58.2%), nonmarried (64.8%), and nonemployed (59.2%).
EFA and CFA
Results of the EFA supported as many as three factors on the basis of the eigenvalue-greater-than-1 rule and scree plots. The eigenvalues of one-factor, two-factor, and three-factor solutions were 4.38, 1.89, and 1.35, respectively. Model fit indices indicated that the three-factor solution demonstrated superior fit (RMSEA = 0.07; CFI = 0.96; TLI = 0.92; AIC = 16,217), accounting for 63.5% of the variance (see Table A.2 in the Supplemental Appendix). Table 2 shows the rotated factor matrix of the three-factor solution: five items from the NIHTB–CB fluid cognition loaded .40 or greater on the first factor (fluid cognition), five from the EFPT on the second (functional cognition), and two from the NIHTB–CB crystallized cognition on the third (crystallized cognition).
Rotated Factor Matrix
Note. Boldface indicates items loading ≥.40.
p < .05.
The results of the CFA indicated that second-order and bifactor models demonstrated superior fit compared with the first-order model (Table A.2). The model fit indices between the second-order and bifactor models were comparable, with a small AIC difference of 1. This indicates that the model fit between the second-order and bifactor models was not statistically significant. On the basis of parsimony, we selected the simpler second-order model as the final model (RMSEA = 0.04; CFI = 0.97; TLI = 0.92; AIC = 16,117; Figure 1). The second-order model supports a hierarchical structure of cognition in which covariation among the three first-order cognitive constructs (i.e., fluid, functional, and crystallized cognition) is accounted for by a higher order common factor (i.e., g factor).
The final model of cognition.
Relationships Between Factors of Cognition and Neuropsychological Tests
As shown in Table 3, factors of cognition were significantly correlated with all neuropsychological tests in the same direction (absolute r = .28–.80, p < .001). Fluid cognition was moderately to strongly correlated with neuropsychological tests that assessed fluid cognition (i.e., episodic memory, working memory, processing speed, and executive function). Crystallized cognition had a moderate-to-strong correlation with neuropsychological tests assessing crystallized cognition (i.e., receptive vocabulary and verbal knowledge) and with the WAIS–IV Letter–Number Sequencing. No moderate or strong correlations were observed between functional cognition and the neuropsychological tests.
Correlations Between Factors of Cognition and Neuropsychological Tests
Note. Boldface indicates moderate to strong correlation (absolute r ≥ .5). BVMT–R = Brief Visuospatial Memory Test–Revised; D–KEFS CWIT = Delis–Kaplan Executive Function System Color Word Interference Test; PASAT = Paced Auditory Serial Addition Test; PPVT–4 = Peabody Picture Vocabulary Test–Fourth Edition; WAIS–IV = Wechsler Adult Intelligence Scale–Fourth Edition; WCST–64 = 64-card Wisconsin Card Sorting Test; WRAT–4 = Wide Range Assessment Test–Fourth Edition.
Lower scores indicate higher performance.
p < .001.
Follow-Up Analyses
To identify if the establishment of functional cognition as an independent construct in the EFA was due to the motor demands of the EFPT, a follow-up factor analysis identified a four-factor solution, accounting for 65.7% of the variance (see Table A.3 in the Supplemental Appendix). Components of the EFPT loaded on a single factor, whereas components of the NIHTB–MB loaded on the other three factors.
Discussion
Cognitive assessment is important when a neurological event has occurred, because test results can guide interventions and help the treatment team provide consistent support to the patient to facilitate learning. Because the person’s behaviors will have changed, it would be important to educate family members to understand that changed behavior is not willful and that they can provide cognitive support to enable their loved ones to have successful performances.
We were particularly interested to learn whether the ecological, performance-based EFPT measured a construct distinct from fluid cognition. Despite the historical perspective that cognitive abilities represent two constructs, crystallized and fluid abilities, our findings indicate that the best-fitting model distinguished three cognitive constructs. As expected, the two established cognitive constructs of fluid cognition (the ability to think and reason abstractly and solve problems) and crystalized cognition (the ability to use skills, knowledge, and experience) defined by Horn and Cattell (1966) were supported. In addition, a third, distinct cognitive construct, which we call functional cognition, was distinguished as a separate component of intellectual functioning referred to as g. We define functional cognition as the processes (i.e., initiation, organization, sequencing, safety and judgment, and completion) supporting the performance of complex everyday tasks and activities in a safe manner. The CFA results demonstrated that three constructs contribute hierarchically to a general cognitive factor. Therefore, it is essential to measure functional cognition to address the performance issues faced by those who experience stroke, head injury, and spinal cord injury. Moreover, the results indicate that the EFPT taps a distinct construct of functional cognition and not fluid cognition, which includes executive function. Therefore, we may consider revising the name of the EFPT to the Functional Cognition Performance Test to reflect better the nature of the construct it assesses.
Because functional cognition involves the performance of a task, we wanted to be certain that the distinct characteristic of the functional cognition evaluation was that performance required cognitive skills and was not due to the motor requirements of the assessment. We examined whether functional cognition could be accounted for by the motor aspects of the task by including measures from the NIHTB–MB in a secondary analysis. The NIHTB–MB did not load on the functional cognition factor and loaded on its own separate factor, supporting the hypothesis that differences in the functional cognition factor from the crystallized and fluid cognition factors are likely due to differences in the underlying cognitive skills but not the motor skills required in performing functional tasks.
The results have important implications for clinical practice. The Montreal Cognitive Assessment (MoCA; Nasreddine et al., 2005) and the Mini-Mental State Exam (MMSE; Folstein et al., 1975) are the prevailing cognitive measures used by clinicians to evaluate cognition and estimate patients’ performances and needs in everyday life (Shen et al., 2016). Like most neuropsychological tests, the MMSE and MoCA assess fluid and crystallized cognition in a controlled context (e.g., copying a figure and repeating a phrase), which contrasts with the complexity and variability of everyday activities (Baum et al., 2008; Wu et al., 2021). Results from neuropsychological tests yield an incomplete assessment of patients’ performances in everyday life situations, leading to suboptimal care transitions and misallocation of health care resources (Depp et al., 2012). Functional cognition assessment should be a routine component of clinical practice to ensure a holistic evaluation of cognition and enhance clinical decision making. In addition to the EFPT, there are other tests of functional cognition and performance of daily tasks. For instance, the Menu Task (Edwards et al., 2019) offers a quick screening of functional cognition and can be used in both acute and postacute settings, whereas the Performance Assessment of Self-Care Skills (Holm & Rogers, 2008) offers a more comprehensive evaluation of the occupational performance of daily life tasks. These or other assessments addressing functional cognition could be integrated into rehabilitation practice to determine the capacity to function safely in community settings and identify people who could benefit from intervention to guide the use of strategies.
Readers should note the limitations of this study. Although we established that the EFPT component scores load on a factor separate from fluid and crystallized abilities, we have not demonstrated that the functional cognition factor predicts performance in real-world settings. Although the EFPT requires participants to perform functionally relevant tasks (e.g., make oatmeal, pay bills, make a telephone call, and take medications), the context of the performance is a laboratory setting with materials and activities that only represent the real world; the individual’s actual home environment and materials are not used. Moreover, we have not demonstrated that functional cognitive assessment is more predictive of daily life performance than neuropsychological tests. How functional cognition is affected or improved by cognitive interventions is yet to be investigated.
Research Implications
The findings of this study challenge the assumption that performance is predicated on cognitive abilities as measured primarily by neuropsychological tests, such as executive function and working memory. Functional cognition is a distinct construct and the cornerstone of people’s ability to perform. Therefore, we need to embrace functional cognition assessment in research and clinical practice as central to our roles as occupational therapists and scientists who support people as they engage in meaningful daily activities. Future research should identify the optimal approaches for measuring and intervening in disorders of functional cognition. This research agenda could include the development of new tools or clinical practice guidelines. Researchers may want to investigate the consequences of impaired functional cognition on daily functioning and other important outcomes (e.g., participation, mood, and quality of life). Our research will seek to establish the generalizability of EFPT results to other performance-based assessments and clinical populations. Occupational therapy scientists have the opportunity to describe the role of functional cognition in other areas of complex daily activities, such as work, driving, and community living, and to describe the translation of functional cognition to community and environment.
Implications for Occupational Therapy Practice
This study has the following implications for occupational therapy practice: ▪ It is important for occupational practitioners to evaluate clients’ functional cognition routinely and advocate for an improved understanding of the role of functional cognition in complex everyday tasks. Accumulating evidence supports inclusion of these measures in national quality programs to demonstrate our positive impact on clients’ lives. ▪ Assessment and intervention protocols used to measure executive dysfunction should not be used as substitutes for functional cognition.
Conclusions
This study provides important and timely evidence to establish functional cognition as a unique construct that is distinct from executive function. The findings suggest that functional cognition, along with crystallized and fluid cognition, contributes hierarchically to a general cognitive factor. This distinction provides the basis for future research on functional cognition and supports advocacy for the profession’s role in evaluating and treating deficits of functional cognition in routine clinical practice.
Supplemental Material
Supplementary material for Functional Cognition: Distinct From Fluid and Crystallized Cognition?
Supplementary material, sj-pdf-1-aot-10.5014_ajot.2023.050010.pdf for Functional Cognition: Distinct From Fluid and Crystallized Cognition? by Carolyn M. Baum, Stephen C. L. Lau, Allen W. Heinemann and Lisa Tabor Connor in The American Journal of Occupational Therapy
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
