Abstract
Screening tools are needed in occupational therapy practice to determine which clients require formal evaluation. Because screening tools also tell us who does not require formal evaluation, they are key to improved clinical efficiency. Screening tools are brief, easy to administer, and often freely available. These qualities also lead to misuse of screening tools, including using them to measure progress over time or to serve as a confirmatory assessment on which to base treatment planning. We present additional common missteps of screening tool use, including a lack of consideration for a tool’s psychometric properties, and exemplars of these common misuses in adult and pediatric practice. Finally, we offer solutions to address these concerns.
The authors describe common missteps of screening tool use in adult and pediatric practice and offer solutions to address these concerns.
The Occupational Therapy Practice Framework: Domain and Process (4th ed.; OTPF–4; American Occupational Therapy Association [AOTA], 2020) identifies three important steps of the occupational therapy process: (1) evaluation, (2) intervention, and (3) targeting outcomes. Occupational therapy practitioners collaborate with clients, families, and populations during the evaluation to understand their unique strengths and challenges related to desired performance and participation. The OTPF–4 emphasizes that screening for potential difficulties can be a first step in the evaluation process (AOTA, 2020). However, the role of screening in the occupational therapy process is ambiguous, and formal guidance on choosing appropriate screening tools is lacking. Therefore, in this column we aim to (1) discuss the importance of screening terminology and (2) highlight the need for proper use of screening tools in occupational therapy practice.
Importance of Screening Terminology and Methods
Screening provides occupational therapy practitioners with a glimpse of a person’s or population’s potential needs, strengths, limitations, and contexts; however, screening does not inform treatment planning and indicates only whether additional formal evaluation is required (Edwards et al., 2019; Vroman & Stewart, 2014). Although screening is often brief and sometimes a nonbillable function (Kramer & Grampurohit, 2020), the appropriate implementation of screening is vital for determining the likelihood that a client does or does not have a given characteristic (Edwards et al., 2019).
Screening is an initial step in the evaluation process and is used exclusively to determine who needs further evaluation, what areas need to be furthered evaluated, and which evaluation tool should be used next, if appropriate. Screening tools are not meant to be administered more than once, to be used as a measure of progress, or to guide treatment planning.
The current press for productivity and outcomes brings an unprecedented push for efficiency (Kerasidou, 2019). Screening provides a means to enhance clinical efficiency by quickly determining those who do not need further care and thus creating time for those who do. Similarly, in research, screening tools inform selection of study participants. Occupational therapy practitioners often use inclusion and exclusion criteria to understand which clients best match the proposed assessment or intervention. Optimizing screening procedures has future dividends in the form of decreased cost, lower time demands, and enhanced value-based care.
Occupational therapy practitioners consciously and subconsciously use multiple methods to perform screenings, including chart reviews and discussion with clients, families, and colleagues. For example, an occupational therapist in a school setting may briefly observe a child in the classroom or ask the teacher for examples of the child’s work. These methods provide valuable information; however, use of more formal screening tools may be required to detect those who need diagnostic assessment. Edwards et al. (2006) used a formal cognitive and sensory screening battery with an inpatient stroke population and compared their findings with those documented using usual-care multidisciplinary assessment methods. Formal screening methods identified impairments that went undetected and undocumented by the medical care team in more than 90% of the sample. Similarly, in pediatric settings, practitioners can administer specific screening instruments, such as the Bayley Scales of Infant and Toddler Development (Bayley & Aylward, 2019). Practitioners administer screenings on a daily basis across populations and settings. It is time for these efforts to be supported with consistent language and intentional efforts to ease the barriers to proper implementation of screening.
Practitioners often use the terms screening and assessment interchangeably, although each of these processes has its own distinct definitions and purposes (Kramer & Grampurohit, 2020). Occupational therapy practitioners use assessments to predict, identify, or measure occupational performance and underlying performance skills. Unlike screening tools, assessments are traditionally more comprehensive in terms of time and content and require more advanced training. In contrast, screening tools aid in determining which people require additional evaluation and do not provide in-depth, confirmatory findings.
Argument
The lack of recognition of the intent of screening and how it fits within the overall evaluation process paired with the inconsistent use of terminology ultimately leads to misuse of these tools (Table 1). Screening tools inherently have high clinical utility because of their brevity, low cost, and ease of use. They have high sensitivity (i.e., the ability to detect people who might have a given characteristic) and low specificity (i.e., a high rate of false positives). In other words, screening tools cast a wide net so as not to miss anyone with a certain characteristic, and as a result they sometimes wrongly classify people. For this reason, further evaluation is needed. We identify three concerns associated with the current use of screening tools in the clinical and research realms. Practitioners and researchers often overlook or lack awareness of the psychometric properties of screening tools. Moreover, they use screening tools with populations for whom the tool was not developed or with which it was not tested. Lack of acknowledgment of a tool’s psychometric history strongly affects the accuracy of results; thus, negligence in this regard impairs the ability to provide high-quality care. Practitioners and researchers measure constructs during screening that do not necessarily translate to valued occupations and contexts. We acknowledge that at times the screening of certain performance skills may be warranted. However, the need to recognize that improvement in performance skills does not equate to occupational performance remains. Practitioners and researchers use and document results of screening tools as though they are diagnostic assessments. Screening tools are designed to cast a wide net. Diagnostic assessments are designed to more accurately ascertain the presence and extent of a given condition or situation.
Exemplars of Screening Tool Misuse
Collectively, these concerns lead to less-than-optimal information on which to base intervention and discharge planning and ultimately lead to poor care for clients. Comparing screening tools to a car’s check-engine light is an easy way to highlight these concerns. Both a screening tool and a check-engine light serve as brief, easy-to-implement fail-safes to determine when a problem might exist. When a check-engine light comes on, a car requires further expert opinion in the form of a mechanic who confirms the issue (i.e., formal evaluation) and then makes a plan to address the issue (i.e., treatment plan). A person would never trust their check-engine light alone to diagnose a problem with their car or have a mechanic begin working on it without first confirming that a problem exists. Occupational therapy clients deserve the same level of rigor in the evaluation processes. Although these issues are pervasive across settings and populations, we use two exemplars to illustrate the existing shortcomings.
First Exemplar: Bringing Cognition to the Forefront of Rehabilitation
The physical effects of stroke are the dominant focus of early-stage rehabilitation. This emphasis is likely due to the visible nature of these impairments, relative ease of measurement, and the fact that clients seem to be more readily aware of the impact of observable impairments. Higher level cognitive impairments related to instrumental activities of daily living become apparent later in the recovery process when the client reenters the complexities and novelties of the real world (Fleming & Strong, 1999). This imbalance between readily observable physical impairments and less noticeable cognitive factors does not reflect corresponding limitations on function; mild cognitive deficits can become overtly apparent in tasks that demand seamless integration of multiple higher-level processes (Morrison et al., 2015; Spitzer et al., 2011). Screening for and evaluating cognitive impairments appropriately in the early stages of neurorehabilitation are critical.
Occupational therapy practi tioners predominantly use either neuropsychological, paper-and-pencil–type methods or simple observation of task performance as a means of screening for cognitive impairments (Burns & Neville, 2016; Pilegaard, 2014). These measures are not ecologically valid representations of a person’s functional cognitive abilities and do not provide an in-depth assessment of cognitive processes (Lezak et al., 2004; Suchy, 2009). The Mini-Mental State Examination (Folstein et al., 1975) is a commonly used neuropsychological screening tool despite its having questionable psychometric properties, including being no better than chance at detecting mild cognitive impairment (Nys et al., 2005; Toglia et al., 2011). Clients may score within the cognitively intact range on this screening instrument but present with functional cognitive limitations when asked to complete a task with multiple steps and interdependent components (Hartman-Maier et al., 2009; Morrison et al., 2015). As a second example, the Short Blessed Test (Katzman et al., 1983) is a commonly used cognitive screener developed to detect dementia symptoms. Despite not being validated for use with people with stroke, it is widely used as a cognitive screener with this population. The Montreal Cognitive Assessment (Nasreddine et al., 2005) is a better alternative because it has been validated for use with people with stroke and demonstrated sounder psychometrics; however, for those with mild stroke, additional performance-based assessment may still be needed to confirm a lack of cognitive impairment.
Clinical screening for the functional impact of cognition is almost exclusively accomplished through informal observation (Pilegaard et al., 2014). Although the use of clinical reasoning through observation is inextricable from the cognitive evaluation process, it is, in and of itself, inadequate. Finding screening methods that point toward the important factors that will support function is a critical first step to efficient rehabilitation care.
Second Exemplar: The VMI as a Screening and Outcome Assessment Tool for Handwriting
Poor handwriting is one of the most common reasons for referral to school-based occupational therapy services (Pfeiffer et al., 2015). The Beery–Buktenica Developmental Test of Visual–Motor Integration (VMI) is the tool most frequently used with this population (Beery & Beery, 2010; Doyle & Goyen, 1997; Feder et al., 2007). The VMI is a narrowly focused assessment tool used to identify students who are likely to need additional support for visual–motor tasks. Because occupational therapy focuses on participation, the VMI can provide only a small piece of information. However, it is common for the VMI to be used as an outcome measure when handwriting itself is the desired outcome.
A moderate correlation does exist between the VMI and handwriting performance (Brossard-Racine et al., 2011; Parush et al., 2010); however, this relationship appears to be merely correlative and not causative (Goyen & Duff, 2005). This concept was underscored by a study that compared a visual–perceptual skills intervention with a handwriting intervention (Howe et al., 2013). Pre- to postintervention comparisons demonstrated significantly greater improvements in the handwriting group but no notable differences between groups on VMI scores. Similar findings of improved handwriting function in the absence of improved VMI scores have been replicated by others (Bolton et al., 2020; Pfeiffer et al., 2015). Finally, use of the VMI to discriminate between children with and without handwriting difficulties correctly identified only 34% of those with poor handwriting (Goyen & Duff, 2005). In sum, the VMI certainly has a place in occupational therapy practice, but only when used for its stated intended purpose. When a risk exists, occupational therapy practitioners have a responsibility to directly evaluate which participation areas are relevant.
Steps to Positive Change
Several steps may be taken to not only correct for these common assessment missteps but also to strengthen the overall rigor and relevance of occupational therapy evaluation. We provide some suggestions here: Endorse ongoing professional development related to rigorous use of screening tools to stay abreast of this knowledge throughout one’s career. Emphasize the role of screening, and provide direct guidance on its implementation in official documents (e.g., OTPF–4, guidance documents). Support occupational therapy managers in streamlining the appropriate use of screening tools within their settings. Identify a subset of screening tools tailored to the clinical environment (e.g., setting, commonly seen diagnoses, psychometric considerations). Set up an organized workstation where screening materials can be obtained. Organize ongoing seminars to educate occupational therapy practitioners on the use of identified screening tools. Within practice settings, establish professional standards that require practitioners to use screenings appropriately. Recommend that practitioners build a toolbox that links screening tools to commonly used assessments that can answer questions about a person’s capabilities, potential environmental barriers, and occupational choices.
Conclusion
Reflection on current practice patterns and planning and execution of appropriate action steps are necessary to ensure proper use of screening tools in both clinical and research settings. Changes may be best executed through several different levels of care professionals, including rehabilitation directors, occupational therapy practitioners, educators, and researchers. Rigorous use of standardized assessments across the life span produces valid data on which to base client-centered intervention and monitor client progress.