Abstract
Stroke is a leading cause of disability in adults that affects nearly 800,000 people in the United States annually (American Heart Association, 2012). More than 50% of adults with stroke have resulting cognitive impairments such as memory deficits and executive dysfunction (Bour, Rasquin, Boreas, Limburg, & Verhey, 2010). The National Institutes of Health Stroke Scale (NIHSS; Brott et al., 1989) is used to assess certain stroke-specific domains susceptible to impairments, including language, sensory, and motor skills. Mild stroke is defined as a score of <5 on the NIHSS, which indicates minor deficits that may not significantly affect basic activities of daily living (Crespi et al., 2013; MacKenzie, Gould, Ireland, LeBlanc, & Sahlas, 2011; Morrison, Edwards, & Giles, 2015).
As many as 71% of adults with mild to moderate stroke may be discharged to the community without identification of impairments or referral to formal services (Edwards et al., 2006; Mok et al., 2004). Unlike many motor impairments, cognitive deficits are invisible and difficult to detect during hospitalization (Bour et al., 2010). Impairments are more likely to appear during performance in higher level activities of daily living, which typically occur once the client is discharged to home (MacKenzie et al., 2011).
Current Trends in Cognitive Assessments
Home health practitioners engage in a clinical reasoning process when deciding how to assess cognitive performance in adults with mild stroke. Cognition is assessed using screening tools, nonstandardized assessments, and standardized assessments. Standardized assessments can be subdivided into performance-based and non–performance-based assessments.
Cognitive screening tools are typically short assessments that guide practitioners to identify deficits and create an initial profile of a client’s abilities (Hartman-Maier, Katz, & Baum, 2009). A survey that tracked practice trends among occupational therapists treating home health clients with stroke revealed that the Mini-Mental State Examination (Folstein, Folstein, & McHugh, 1975) was the primary standardized cognitive assessment used by 54% of reporting therapists (Moulton, 1997). Cognitive screening measurements can be an effective guide for standardized assessment selection. However, used alone, they are inadequate at providing information on real-world performance deficits (Burgess et al., 2006; Hartman-Maier et al., 2009).
Nonstandardized assessments include skilled observation and facility-developed tools lacking vetted psychometric properties. Skilled observation is a method of nonstandardized assessment that uses the practitioner’s clinical reasoning skills to analyze and interpret occupational performance. A survey study by Pilegaard, Pilegaard, Birn, and Kristensen (2014) revealed that Danish occupational therapists most frequently used nonstandardized assessments such as skilled observation and semistructured interviews to assess cognitive function.
Standardized cognitive assessments entail a protocol for administration and scoring, have evidence of psychometric properties, and provide practitioners with information on impairments (Hartman-Maeir et al., 2009; Morrison et al., 2015). Historically, standardized cognitive assessments used by occupational therapy practitioners consisted of pencil-and-paper tests. However, current evidence suggests that performance-based assessments provide findings that more accurately approximate and predict function in real-world settings (Baum et al., 2008; Bottari & Dawson, 2011; Burgess et al., 2006).
A survey of standardized executive function assessment use in Canada revealed that although standardized tools were widely used when detecting stroke-related impairments across the continuum of care, practitioners in community-based settings rarely used standardized assessments that approximated real-world impairments (Korner-Bitensky, Barrett-Bernstein, Bibas, & Poulin, 2011). Assessments that measure real-world performance have been termed ecologically valid. Ecologically valid assessments are conducted within the context of real-world activities, providing a measure of performance within real contexts (Burgess et al., 2006; Jovanovski et al., 2012). The argument for use of assessments with ecological validity is that they approximate reality, unlike many other well-structured assessments. Traditional pencil-and-paper tests do not measure true goal-directed behaviors that involve concurrent performance of skills such as multitasking, mental flexibility, and divided attention, which occur routinely in daily occupations (Baum et al., 2008; Burgess, Veitch, de Lacy Costello, & Shallice, 2000).
Cognition in Home Health Care
Home health occupational therapy practitioners provide a necessary service for adults who are homebound and unable to continue therapy at an outpatient clinic. Practitioners in this setting are afforded the opportunity to evaluate cognition using nonstandardized and standardized assessments within a naturalistic context. However, a knowledge gap exists regarding how these practitioners assess cognition in home health care.
Prior studies revealed that skilled observation is the primary means of assessing cognition; however, no recent studies have directly addressed cognitive assessment trends in home health care (Korner-Bitensky et al., 2011; Pilegaard et al., 2014). The purpose of this study was to survey occupational therapy practitioners working in home health care to determine what cognitive assessments they used and how their confidence and productivity demands affected assessment use for adults with mild stroke.
Method
Research Design
This study used a researcher-developed survey. Results are presented using descriptive statistics. The study was approved by the Texas Woman’s University institutional review board.
Instrument
The principal investigator (Suzanne C. Burns) developed a self-administered survey after an extensive literature review examining cognitive assessments used for people with stroke. This survey built on prior surveys on home health practice and cognitive assessment use (Moulton, 1997; Pilegaard et al., 2014). In addition, the survey explored confidence and productivity demands. Confidence was defined as self-efficacy in cognitive assessment administration, and productivity was defined as visit duration, number of visits completed each day, and drive time requirements. The survey was designed to take <10 min to complete and comprised of 42 questions that included 11 demographic questions, 2 yes–no questions to determine whether the respondent assessed cognition in home care clients, a list of 20 assessments to be rated on a Likert scale according to use, 2 multiple-select questions on administration methods, 3 questions about confidence rated on a Likert scale, 2 questions on the influence of productivity demands rated on a Likert scale, and 2 open-ended questions to allow elaboration on other assessments used and any assessment training attained.
On the basis of the literature review, the survey listed, in no particular order, 20 cognitive assessments (nonstandardized, screening, and standardized performance-based) to determine frequency of use in practice. Frequency of use was rated as 0 = never heard of, 1 = never used, 2 = sometimes used, 3 = frequently used, or 4 = standard of care (SOC). This study defined standard of care as a diagnostic process that a clinician always follows with this population. Table 1 lists the assessments by category and the percentage of respondents who used each assessment as the standard of care or frequently.
Categorized Assessments and Use by Participants (N = 56)
Note. ADL = activities of daily living; OT = occupational therapy.
Two expert clinicians evaluated the survey for content validity. Feedback resulted in format, wording, and organization changes. The survey was piloted with three occupational therapists who worked in home health care in a large metropolitan area. Each had at least 3 yr of experience in home health care and with adults poststroke. One of the clinicians suggested adding the Short Blessed Test (Katzman et al., 1983) to the list of assessments, bringing the total number of assessments to 20.
Participant Selection
Participants were identified by the American Occupational Therapy Association (AOTA) Home and Community Health Special Interest Section (HCHSIS) and through AOTA’s OTConnections Web-based discussion forum. Of 501 contacts sent by the HCHSIS, 240 were randomly selected using a random number generator. Survey respondents (n = 60) included both occupational therapists and occupational therapy assistants. Four surveys were excluded because of incomplete data, leaving a final sample of 56 (54 occupational therapists, 2 occupational therapy assistants). All clinicians met the established inclusion criteria: They (1) were employed in adult home health, (2) resided in the United States, (3) were literate in written English, and (4) currently practiced as an occupational therapist or occupational therapy assistant.
Data Collection
The survey was made available in two ways. First, a paper invitation was mailed to the 240 randomly selected practitioners across the United States with instructions on accessing the survey electronically through PsychData.com. In addition, the invitation and link were posted 3 times on the OTConnections online forum, and the survey was made available for 30 days. The survey was anonymous, and participant information was not linked to the tool.
Data Analysis
Data were analyzed using IBM SPSS Statistics (Version 22; IBM Corp., Armonk, NY) and Excel (Microsoft, Redmond, WA). Descriptive statistics were used to determine frequency of use of each assessment. The cognitive assessments were categorized into nonstandardized assessments, screening tools, and standardized performance-based assessments on the basis of a literature review (Hartman-Maeir et al., 2009; Poulin, Korner-Bitensky, & Dawson, 2013).
Results
Participants
Fifty-six respondents (age range = 27–68 yr) completed the electronic survey. Fifty-five percent (n = 31) had a master’s or doctoral degree, 43% (n = 24) had a baccalaureate degree, and 1 participant had an associate’s degree. The participants’ years of experience in home health care varied. Five percent (n = 3) of the participants had <1 yr of experience, 52% had 1–10 yr (n = 29), and 43% (n = 24) had ≥11 yr. All of the participants had substantial work experience with adults with stroke; in fact, 75% (n = 42) had ≥11 yr of work experience with this population. Nearly one-third (30%, n = 17) of respondents had a certification, and 16% (n = 9) had specialized training in cognitive assessment use.
Assessment Use
Ninety-six percent (n = 54) of this sample of home health occupational therapy practitioners reported assessing cognition. To determine the frequency of use of the different cognitive assessments, we asked participants to rate assessment use on the previously mentioned 5-point scale. They used assessments categorized as nonstandardized as an SOC most often (85%). Frequently used assessments included standardized performance-based (31%), screening (40%), and nonstandardized (28%). The percentages of assessment use by category are shown in Figure 1.
Frequency of use of cognitive assessment types in home health care.
Influences on Assessment Use
Participants responded to five questions about the influence of confidence and productivity demands on the reasoning process in selecting cognitive assessments. The three questions about confidence levels revealed that between 14% and 22% of participants were either not confident or somewhat not confident with selecting cognitive assessments, assessing cognition, and designing cognition-focused treatment plans. In addition, 47% saw themselves as somewhat confident, which may be interpreted as lacking some degree of confidence in assessment use and treatment planning around cognitive impairments (Figure 2). Respondents reported that productivity demands significantly influenced (28%) or moderately influenced (26%) cognitive assessment selection. In addition, they reported that productivity demands significantly influenced (10%) or moderately influenced (30%) whether they used a cognitive assessment to evaluate cognition (Figure 3).
Participants’ confidence in their ability to select assessments, assess cognition, and use assessment findings to design treatment plans (N = 56).
Participant ratings of the influence of productivity demands on cognitive assessment selection and use for adults with mild stroke (N = 56).
Data were analyzed to determine whether assessment use differed for respondents with specialized training (16%). Those with specialized training showed 15% greater use of standardized performance-based assessments than participants without formal training. A secondary analysis completed on participants who used screening tools showed that 96% of these participants also used standardized assessments at least sometimes.
Discussion
Participants reported that when evaluating cognition in adults with mild stroke, 85% selected nonstandardized assessments as the SOC and 28% selected nonstandardized assessments frequently. The findings of this study are consistent with those of other surveys in that the most widely used methods in cognitive assessment are nonstandardized. However, nonstandardized assessments, including skilled observation of routinized tasks may not be entirely accurate at capturing the breadth and depth of cognitive impairments (Edwards, Hahn, Baum & Dromerick, 2006).
Confidence has been recognized as a possible barrier to evidence-based practice in stroke rehabilitation professionals (Pollock, Legg, Langhorne, & Sellars, 2000). Like the study by Pollock et al. (2000), our study found that between 31% and 39% of respondents were confident in using cognitive assessments but that the majority of respondents expressed a lack of confidence to some degree. It is possible that the respondents relied more heavily on nonstandardized assessment methods because they lacked confidence in selecting, using, and interpreting standardized cognitive assessment tools.
Skilled observation is a nonstandardized method of evaluating real-world performance in a naturalistic context. However, observers may evaluate only a small component of cognition, such as procedural memory while performing daily routine activities, instead of higher level cognitive skills that appear in ecologically valid assessments with verisimilitude. Assessments with verisimilitude involve cognitive demands similar to those in the naturalistic environment and may have characteristics such as multitasking or interruptions that can more accurately capture cognitive impairments such as executive dysfunction (Burgess et al., 2006).
It has been suggested that productivity requirements, including lack of time and large workloads, may influence evidence-based practice (McCluskey, 2003). Similarly, our study found that home health–specific productivity demands influenced assessment use. More than 50% of participants reported that productivity had a moderate or significant influence on what assessment they used to evaluate cognition. Moreover, 40% of the respondents reported that productivity demands moderately or significantly influenced whether they assessed cognition.
Respondents reported frequent use of standardized performance-based assessments (31%) and screening tools (40%). In fact, 96% of respondents who reported screening tool use also reported use of standardized assessments at least sometimes. Cognitive evaluation recommendations support this practice, and screening tools can be used as a precursor to general measures of cognition or cognitive tests for specific domains (Hartman-Maeir et al., 2009). In addition, participants who had continued training in cognitive assessments were more likely to use standardized assessments.
Implications for Occupational Therapy Practice
This study’s findings have the following implications for occupational therapy practice:
Occupational therapy practitioners should seek out psychometrically sound cognitive assessments and continuing education opportunities. Those with specialized training are more apt to use valid and reliable instruments.
Guidelines and evidence need to be developed regarding the most appropriate assessments for the home health practice setting.
Ecologically valid assessment use in the home environment offers a promising way to evaluate cognition reflecting real-world performance in context.
Limitations and Future Research
A direct contact approach was not used for data collection, and results should therefore be interpreted with caution. Selection and nonresponse biases may be present because the survey likely attracted respondents who were interested in cognition, and those not interested in the topic may have been less likely to participate. These biases may have affected the findings because the respondents may have had more experience and knowledge about cognition than other practitioners. The Kitchen Task Assessment (Baum & Edwards, 1993) and Execution of a Cooking Task (Chevignard et al., 2008) were reported as the most frequently used standardized performance-based assessments; however, it is unclear whether respondents used the assessments or interpreted the assessment names as general skilled observation in the kitchen. Lastly, mild stroke was not defined on the survey, so respondents’ interpretation of the definition may have been inconsistent.
Future research in this area should explore how occupational therapy practitioners are influenced by productivity requirements. Research that guides the development of assessment guidelines in home health care should also be explored.
Conclusion
Skilled observation of task performance was the most frequently used method of cognitive assessment in this study. Practitioners who used screening tools also used standardized assessments at least sometimes. Standardized assessments were more likely to be used if the practitioner had additional training in cognitive assessments. This study also confirmed that confidence and productivity requirements influence clinical reasoning with regard to use of cognitive assessments.
Footnotes
Acknowledgments
We thank the practitioners who contributed to this study by completing the survey.