Abstract
An increase in the older adult population will likely be associated with an increased need for long-term-care communities, such as assisted living. A primary goal of assisted living communities is to help residents maintain their health and well-being. To accomplish this goal, direct care workers employed in these settings are responsible for monitoring residents for cues that might signal problems and then responding appropriately. However, very little is known about these workers’ decision making. To gain a better understanding, direct care workers from assisted living facilities took part in a critical incident interview and a scenario-based interview. The interview data revealed various cues that were categorized as cognitive, physical, or emotional in nature. Specific explanations for the scenarios were primarily that the situation was the result of a cognitive/emotional/social issue or a physical health issue. The type and number of explanations varied widely from scenario to scenario. Of the actions participants described taking to handle the scenarios, gathering and using information was discussed more than any other action. This research has implications for training of formal and informal caregivers and also highlights the potential need for decision support systems in this domain.
Introduction
The aging of the population is increasingly apparent in the United States. In 2012, approximately 41.5 million individuals, or 13.4% of the U.S. population, were 65 years and older (U.S. Census Bureau, 2012). It is estimated that by 2030 the proportion of older adults in the United States will increase to 20% (U.S. Census Bureau, 2014). Older adults reside in various forms of housing, including living in their own home or apartment, residing with family members, or choosing a residential long-term-care community (e.g., independent living, assisted living, or nursing home). See Table 1 for definitions of these various long-term-care communities.
Definitions of Long-Term-Care Communities
Note. Definitions from Bookman, Harrington, Pass, and Reisner (2007). ADLs = activities of daily living.
One of the primary factors in how older adults or their families select a particular form of supportive housing is the extent of an older adult’s care needs. Caffrey et al. (2012) reported that 38% of residents living in residential care facilities received assistance with three or more activities of daily living (ADLs), such as feeding oneself, bathing, dressing, or grooming (Lawton, 1990). Thirty-six percent received assistance with one to two ADLs, and only 26% received no ADL assistance.
In addition to ADL assistance, many older adults require help with instrumental ADLs (IADLs), such as housework, preparing meals, or taking medications as prescribed (Lawton, 1990). These IADLs include several tasks important to managing chronic health conditions. Recent reports have documented that 50% of residential care community residents had been diagnosed with two to three chronic conditions, and 26% had been diagnosed with four to 10 chronic conditions (Caffrey et al., 2012). Among older adults residing in residential care communities, the 10 most common chronic conditions included high blood pressure, Alzheimer’s disease or other dementias, heart disease, depression, arthritis, osteoporosis, diabetes, chronic obstructive pulmonary disease and allied conditions, cancer, and stroke (Caffrey et al., 2012).
For older adults who need some level of assistance but also want to maintain a sense of independence, assisted living facilities have become a popular option. Assisted living facilities are often viewed as occupying a middle ground between receiving care at home from informal or formal caregivers and admission to a nursing home, which is typically considered a last resort. Ball et al. (2004) suggested that assisted living facilities play an important role in residents’ health care and health promotion as a means of managing decline in residents’ health and well-being. Nunnelee and Gilliland (2001) proposed that the assisted living community should be invested in “health promotion, prevention of illness and injury, maintenance of function, and prevention and exacerbation of the residents’ chronic conditions” (p. 50).
To achieve these goals, assisted living communities employ a variety of staff. A recent report of full-time-equivalent employees working as nursing staff in residential care communities estimated 7.6% were registered nurses (RNs), 10.2% were licensed practical or vocational nurses, and 82.1% were aides. This last category of nursing staff may include unlicensed assistive personnel, nursing aides, nursing assistants, certified nursing assistants, home health aides, home care aides, personal care aides, and personal care assistants (Harris-Kojetin, Sengupta, Park-Lee, & Valverde, 2013). These aides compose the majority of caregiving staff and are on the forefront of providing care and support to residents and are also known as direct-care workers.
Direct Care Workers
Direct care workers have become an integral component of various health care entities, including assisted living facilities. Historically, much of their work was performed by more skilled nursing staff, such as RNs or licensed practical nurses. However, recent decades have seen dramatic shifts in health care, including the rise of managed care, staff shortages, and economic challenges (Ballard & Gould, 2001; Norrish & Rundall, 2001). As a result, many health care facilities have restructured their workforce by increasing the use of direct care workers and shifting the role of nurses to a supervisory position (Ballard & Gould, 2001).
Approximately 70% to 80% of the paid, hands-on care provided to persons with disabilities or chronic care needs is provided by direct care workers (Paraprofessional Healthcare Institute [PHI], 2011). In assisted living, such care includes assistance with ADLs, recreational activities, medications, meal service, and sometimes housekeeping tasks (Ball et al., 2000; Hawes, Phillips, & Rose, 2000). The interaction direct care workers have with residents is typically more substantial than the interaction had by nurses. A study of residential care communities showed that direct care workers spent 8 times the number of hours per resident per day compared to RNs (i.e., 2.16 hr compared with 0.27 hr; Harris-Kojetin et al., 2013). Due to the time spent and level of familiarity they can potentially develop with residents, these direct care workers are in a prime position to monitor residents for cues that might signal problems or concerns, many of which may be treatable and therefore critical to optimizing resident care and quality of life.
The training and education requirements for direct care workers in assisted living is state regulated and therefore varies across the United States. Although direct care workers are not required to be licensed or certified, many are certified, most commonly as certified nursing assistants (CNAs). For instance, an analysis of 717 direct care workers revealed that 42% were CNAs (Ball & Perkins, 2010). Instead, most states mandate only some type of preservice training, which is often fairly minimal in length (Ball, Hollingsworth, & Lepore, 2010). Hawes et al. (2000) found that for unlicensed direct care workers, 75% were required to attend some type of preservice training or orientation, which usually lasted between 1 and 16 hr. However, only 11% completed this training before the start of work. Most training tends to include a period of “shadowing” a more experienced worker (Ball et al., 2010).
Although a critical part of the care network in assisted living, a recent sample of 400 direct care workers in Georgia reported their median hourly pay to only be $8, resulting in average annual net pay of approximately $16,000 (Lepore, Ball, Perkins, & Kemp, 2010). Further, direct care workers tend to exhibit a high level of turnover. Median turnover rates for direct care workers in assisted living have been estimated to be 36.4% for resident caregivers and 13.6% for CNAs (National Center for Assisted Living, 2015).
Direct Care Worker Decision Making
Direct care workers, who are on the front line of caregiving in assisted living, are in a position to detect any type of abnormality or problem with a resident and then to decide whether it requires attention and action. This task is critical to ensuring high levels of care and quality of life among assisted living residents, but making these types of decisions in assisted living settings may involve uncertainty, complexity, and high stakes. Therefore, this task is well suited to explore using a naturalistic decision making (NDM) perspective: “to understand how people make decision in contexts that are meaningful and familiar to them” (Lipshitz, Klein, Orasanu, & Salas, 2001, p. 332). This approach expanded the conceptualization of decision making to include situation recognition and also the generation of appropriate actions (Klein, 2008). In assisted living settings, rapid recognition of certain situations can often lead to more timely and effective solutions, whereas failures or delays in the detection of certain problems may exacerbate their consequences dramatically. For instance, the efficacy of stroke treatment depends largely on how soon treatment is administered after the first symptoms are detected.
Although various lines of research have explored how health care professionals, such as nurses, engage in this type of clinical decision making (Cesna & Mosier, 2005; Cioffi, 2012; Crandall & Getchell-Reiter, 1993; Currey & Botti, 2003; Kazi, Pop, Durso, Ryan, & Cunningham, 2011), there has been little investigation into how direct care workers make decisions regarding potentially problematic situations with assisted living residents. Given the high level of direct interaction with residents, as well as the profile of direct care workers, who often have little training or formal education and exhibit fairly high turnover, this group represents a very different population than traditionally studied in NDM research. There may be critical differences between the decision making of direct care staff and nurses, who have more often been the focus of study. For instance, because workers in assisted living do not have clinical training comparable to RNs, their interpretation of cues may not result in a diagnosis but may be more vague, such as a concern that “something is wrong.” This assessment, although less specific, is still the necessary first step toward taking appropriate action if needed. Therefore, understanding how workers in assisted living make decisions is a worthwhile endeavor.
Authors of one relevant line of research examined nursing assistants who were employed by a community care organization that provided nursing home care and home health care (Tingström, Milberg, & Sund-Levander, 2010). The goal was to understand nursing assistants’ perceptions of signs and symptoms of infection among older adult care recipients. Through focus groups, they identified that signs of infection generally fell into two categories that they labeled person is not as usual and person seems ill. Signs categorized as person is not as usual included exhibiting discomfort, lack of inhibition, aggression, restlessness, confusion, tiredness, and decreased eating. The category of person seems ill contained both general signs (e.g., fever, pale) and specific signs (e.g., cough, wheezing) of illness as well as pain. The authors concluded that the range of cues detected by nursing assistants is evidence of their keen observational ability, although they also admitted that the validity of the mentioned cues remains to be confirmed. Nevertheless, direct care workers are engaging in some level of situation recognition as depicted in NDM.
In an elaboration of their original study, Sund-Levander and Tingström (2013) described nursing assistants’ clinical decision-making process. Their model elaborated on Carroll and Johnson’s (1990) model of decision making, which was not specific to any particular domain. New additions to the model expanded the concept of information search into various strategies used to gather and evaluate information as well as other factors that influence the action ultimately selected by nursing assistants who suspected an infection might be present.
Work by Hawes et al. (2000) investigated whether assisted living staff correctly interpreted various symptoms or cues, such as incontinence, as not part of normal aging and hence cause for concern. This task was similar to the situation recognition task described previously. They found that the majority of staff, which included direct care workers as well as administrators and supervisors, had extensive misconceptions regarding normal aging. Figure 1 depicts that the majority of staff believed confusion, incontinence, depression, and anger to be typical of normal aging, which they are not (Klausner & Vapnek, 2003; Saxon, Etten, & Perkins, 2015; Singh & Upadhyay, 2014). Only 8% of the staff indicated none of these systems was part of normal aging, indicating critical gaps in staff’s understanding of aging, which may impede effective decision making.
Percentages of staff who viewed item as being associated with normal aging, from Hawes, Phillips, and Rose (2000).
In another portion of Hawes et al.’s (2000) study, staff were provided with “care vignettes” that described relatively common problems in caregiving. Each vignette offered a set of potential responses, and staff had to choose either the best answer or all that applied, depending on the vignette. For certain care situations, there was a high level of consistency and accuracy across responses. For example, 95% of staff reported that if a resident developed hives after starting a new antibiotic, the resident’s physician needed to be contacted. However, 15% of respondents also indicated that a resident with incontinence should cut back on the amount of liquids consumed, an approach that could result in other problems, such as dehydration. These responses indicate that a subset of the staff may have worrisome misconceptions regarding urinary incontinence. It may be the case that so many individuals correctly stated they would contact a physician because it is an organizational norm or standard, not because they truly understand that urinary incontinence may be symptomatic of other, treatable problems. However, Hawes et al.’s participants were given a predetermined set of care vignettes and a set of response options to choose from, with very little opportunity for staff to describe what other relevant decisions they may have made or why they made the choices they did, which is an important component of NDM research.
Current Aims
Detecting problems and deciding on appropriate responses are tasks required of individuals in many dynamic task situations (Klein, Pliske, Crandall, & Woods, 2005; Zsambok & Klein, 1997). These processes are also an essential aspect of caregiving in assisted living settings but have yet to be fully investigated with regard to how direct care workers perform these processes. Few studies have focused specifically on assisted living versus nursing homes, which have more rigorous training requirements in place for aides (PHI, 2014) and typically have a higher percentage of registered or licensed nursing staff employed (Harris-Kojetin et al., 2013). The few studies that explored assisted living examined all staff, including administrators and managers who do not engage in day-to-day caregiving of residents, which clouds our ability to understand the decision making of direct care staff specifically.
Therefore, the purpose of the current study was to improve understanding of decision making among direct care staff working in assisted living settings. More specifically, the aims of the study included identifying the cues that initiate the decision-making process and understanding how direct care workers interpret and respond to caregiving situations. This goal allowed for the opportunity to apply NDM techniques to a population that has not yet been represented in NDM research but, given the stakes and complexity, is worthy of study.
The study consisted of multiple qualitative interview methods. The first phase of the study used a critical incident interview technique with the primary goal to elicit a range of decision-making incidents from participants’ experience working in assisted living. This interview approach was derived from the critical decision method described by Klein, Calderwood, and MacGregor (1989). This method uses a set of probes to study the cognitive bases of judgment and decision making in naturalistic settings. This methodology has been successfully used in numerous studies across various professional fields, ranging from fireground command (Klein, Calderwood, Clinton-Cirocco, 1986) to nursing in neonatal intensive care units (Crandall & Getchell-Reiter, 1993). The second component of the study was a scenario-based interview. Using this technique, we presented all participants with a series of hypothetical care situations that systematically differed on a number of characteristics. Similar approaches have been used in previous research on nurses’ clinical decision making (Cesna & Mosier, 2005).
Method
Participants
Thirty-two direct care workers were recruited to participate in the study. Participants were selected to represent two distinct levels of experience, with experience operationally defined as the total time (e.g., years, months) spent working in assisted living facilities, including their current job. A low-experience group was recruited with participants who had between one and 16 months of experience, and a high-level-of-experience group included participants with 3 or more years of experience. Overall, participants’ experience ranged from 2 months to 17.6 years, with a standard deviation of 57.7 months. Differences in responses to the critical incident interview and scenario-based interview between the two experience groups were investigated in the first author’s dissertation research (Bowman née McBride, 2014). However, experience differences were minimal; thus the groups were combined in the current paper.
All participants were currently employed as a direct care worker in an assisted living community for older adults. Participants were recruited only if they worked a minimum of 20 hr per week. Due to the limited interaction with residents, individuals who worked only during the overnight shift were excluded from participating, as were individuals working solely in memory care or dementia units due to the specialized nature of their training. Study participation ranged from 45 min to 2.5 hr, and participants were compensated monetarily for their time. Participants’ demographic information is presented in Table 2. The research protocol was approved by the Georgia Institute of Technology Institutional Review Board, and informed consent was obtained from each participant.
Participant Demographic Information
Recruitment
A list of assisted living facilities was generated using a database managed by the Georgia Department of Community Health, Healthcare Facility Regulation Division. The database was searched for communities listed as personal care homes and assisted living facilities located in the Atlanta metropolitan area. Facility administrators were contacted and the purpose of the project explained. If the facility agreed to be involved, a member of the research team used a variety of tactics to recruit participants, including announcing the study at staff meetings or in-services, visiting the facility at shift change to tell workers about the study on their way out, or providing recruitment flyers to the administrator to distribute. Administrators did not provide any kind of list of employees who could be approached. Additionally, an advertisement for the research study was placed on the Atlanta Craigslist.org website and included the same information as the recruitment materials that were distributed at the facilities.
Individuals interested in participating were screened via a phone call prior to participation. Screening ensured participants met the inclusion criteria described previously (e.g., fit into one of the experience groups, did not work solely overnight shifts or in dementia units). In total, participants came from 19 different facilities. A cutoff of six participants from any single facility was used; 11 of the facilities had one worker participate, six facilities had two, one facility had four, and one facility had five workers participate.
Materials
Interview script
The interview script consisted of two primary components: a critical incident interview and a scenario-based interview. The script is briefly described here but is available from the authors.
Critical incident interview
Participants were asked to recall specific incidents during which they experienced concern for a resident’s health or well-being. In addition to describing the incident, participants were asked about the conditions surrounding the event, including the cue(s) that caused them to become concerned, their interpretation of the situation, how they responded to the concern, and what facets of knowledge were drawn upon during the incident.
Scenario-based interview
This interview method employed a set of scenarios that were used for all of the participants and thus was a basis for comparison across all participants that would not have been possible with the critical incident interview due to the variability of the reported incidents. Seven scenarios were used to represent hypothetical residents exhibiting a range of behaviors and symptoms (see Table 3). Each scenario was presented to the participant and served as the basis of discussion for a series of questions designed to elicit how the participants would interpret and respond to these cues presented in the scenario.
Scenario Descriptions
The goal of scenario development was to create scenarios that included cues that were subtle and nonspecific enough that they could relate to numerous health/well-being issues that commonly occur among older adults. Further, the cues presented in each scenario ranged both in terms of type and number. Some presented only a single cue; others included two cues. Several of the cues were physical in nature, whereas others were related to the cognitive functioning of the hypothetical resident, and the cues in one scenario depicted behavior. The specific cues were selected because they could be considered likely or commonly occurring among an assisted living population. Further, the scenarios were developed without an explicit mention of how long the cue had been present (such as “starting last week”) or whether the cue was atypical for the resident, which placed the onus on the participant to consider that the cue may mean different things depending upon whether it is the first occurrence or a repeated issue for the resident.
The first scenario depicted in Table 3, dinner complaint, was designed to reduce demand characteristics that might be present in the scenario-based interview, that is, to represent a relatively innocuous situation not expected to warrant concern from participants, setting the stage for them to feel comfortable responding to the remaining scenarios. The subject of the neutral scenario was female for half and male for the other half of participants. The remaining six scenarios were written so that half included males and the other half included females.
Demographic and experience questionnaire
A demographic and health questionnaire was used to gather information such as participants’ age, gender, level of education, and ethnicity. The demographics section of the questionnaire was adapted from a preexisting questionnaire (Czaja et al., 2006). The questionnaire also included questions on participants’ job experience, current job duties, resident assignment, certifications, training, and continuing education. These data were used to describe the participant sample.
Knowledge-of-aging questionnaire
To assess participants’ knowledge about characteristics of “normal” aging, a measure of aging and health conditions common among older adults was developed. The assessment was based on Towner’s (2006) Self-Assessment of Geriatric Knowledge. Due to the anticipated education of the current study participants, the assessment was reduced to 24 items, with questions relating to advanced nursing topics removed. Data analysis revealed low performance on this assessment (percentage correct, M = 31, SD = 12.52). Additionally, a split-half reliability analysis produced a Spearman-Brown coefficient of only .53 (.80 is considered reliable). Therefore, these questionnaire data will not be discussed.
Equipment
The interview portions of the study were digitally recorded. Following the interview, these audio files were transferred to a computer and renamed using the appropriate participant identification code.
Procedure
After participants gave informed consent, the goals of the study were discussed with them, and any questions were answered by the interviewer. The interviewer began the critical incident interview by asking participants to recall incidents in which they were concerned about a resident in his or her care, and each incident was discussed. This procedure was repeated for all the incidents the participant reported or until 30 min had passed.
To avoid the risk that the content of the scenarios influenced the type of incidents the participant recalled, the scenario-based interview was always conducted after the critical incident interview. To begin the scenario-based interview, participants were told that they would be given a hypothetical situation describing an older adult living in assisted living and would be asked questions about what they think and might do in that situation. A printed version of the scenario was placed in front of the participant and remained there for the duration of the discussion for that scenario. The interviewer read the scenario aloud, then began the interview by asking questions, such as, “Would this cause you to be concerned?” and “What would your concern be in this case? What do you think might be going on?” The redundancy in presenting the scenario was designed to reduce demands on literacy, vision, and working memory. The interviewer continued through the script, and this procedure was repeated for each scenario.
The order in which scenarios were presented was counterbalanced, except for the first scenario, which was always the dinner complaint (neutral) scenario. The remaining six were counterbalanced using pseudorandom orders. A list of random orders was generated and then reduced to a final set of eight orders using two criteria: (a) no more than two scenarios with two cues (i.e., physical–physical) could occur in a row, and (b) each scenario must be presented in the first and last position at least once. Each order was used with four participants.
Following completion of the interview portion of the study, participants completed the demographic and experience questionnaire, followed by the knowledge-of-aging questionnaire. Upon completion, participants were debriefed, compensated, and thanked.
Results
Data not discussed here can be found in McBride (2014).
Experience, Training, and Job Characteristics
Data from the demographic and experience questionnaire provided insight into work experience, training, and job responsibilities. All but three participants reported experience working with and caring for older adults in at least one other setting, including home health care (78%), nursing home/skilled nursing facility (28%), retirement community (16%), senior center (16%), and hospice care (9%). Regarding training, 78% of participants were CNAs; 9% reported no current health care certifications, licenses, or registrations.
Job characteristics and duties are presented in Table 4. On average, participants reported working approximately 38 hr per week and caring for about 14 residents during their shift. Most frequently reported duties included serving meals, providing personal care, laundry, light housekeeping, companionship, and assisting residents with social or recreational activities. These data provide a profile of the participants and context for the results.
Participant Job Characteristics and Duties
Critical Incident Interview Results
Participants were asked to recall specific incidents during which they experienced concern for a resident’s health or well-being. These incidents provided insight into how participants used cues present in their interactions with residents to detect and respond to concerns regarding resident well-being. To analyze these cues, the audio recordings of the critical incident interviews were first transcribed verbatim. The transcripts were divided among two coders: the first author, who coded eight of the transcripts, and a trained research assistant, who coded the other 24 transcripts. Each person coded units of text containing reference to a cue, which was operationalized as anything that led participants to become concerned about the resident described. These cues were primarily observations made by the participants but in some cases were based upon information reported to the participant by a resident (e.g., “I’m in pain”).
Coding of the transcripts was primarily data driven; coding began with only high-level categories (e.g., cognitive, emotional) that were subject to change as coding progressed. Each time a cue was identified in the transcript, a new code was created to capture it unless one already existed. After coding all transcripts, the full list of codes was examined. Similar codes were combined and some codes were moved from one category to another.
In total, participants described 222 cues across 61 incidents. All participants were able to describe at least one incident, even those with only a month or two of experience working in assisted living. The number of incidents each participant described ranged from one to five, with approximately 80% of participants describing one or two incidents. The total number of cues provided by each participant ranged from one to 36 (see Figure 2). No participant contributed more than 12 cues except for one participant, who described 36 cues across five scenarios. Overall, participants described an average of 3.64 cues per incident (SD = 2.15), suggesting that they were able to recall cues relevant to these events in detail rather than recollecting only a vague sense that something was wrong with the resident.
Histogram of the number of cues reported by each participant.
Participants’ comments regarding what cues prompted them to become concerned were divided into general and specific categories. General cues included comments made by participants that were vague in nature and described a change in the resident at a high level (e.g., she seemed sick) without reference to what was actually being observed. Specific cues were coded as cognitive, physical, or emotional (see Table 5). Chi-square goodness-of-fit tests were used to determine if cues were reported with an equal frequency between the general and specific categories as well as between cognitive, physical, and emotional. The results showed that the cues were not equally distributed, with significantly fewer general cues (5%) and significantly more specific cues (95%), χ2(1, N = 222) = 180.18, p < .001. Additionally, within specific cues, the cues were not equally distributed, with fewer cognitive cues (13.7%) and more physical cues (59.7%), χ2(2, N = 211) = 71.27, p < .001.
Cues Described by Participants During the Critical Incident Interview
p < .05 (i.e., high-level category is significantly different from an equal distribution).
p < .05 (i.e., subcategory is significantly different from an equal distribution).
The distribution of cues presented in Table 5 demonstrates that participants were capable of describing the cues they had previously witnessed in specific terms. Only 5% of the cues were described in general terms, including reports such as “. . . this particular day, I knew he was a little abnormal, the way he was acting.” These general comments included statements about residents declining (in terms of health presumably), seeming to be sick or not feeling well, or seeming different or abnormal. It is possible that the cue was actually more specific at the time of the incident, but due to memory limitations, it could not be elaborated on beyond these general descriptions. However, even if the cue was just a general sense that something was “off,” that sense may still be enough to elicit concern from a direct care worker.
Because the overwhelming majority of cues were described in specific terms, this category was examined in greater detail. Table 6 displays the cues belonging to the cognitive, physical, and emotional categories. Of these categories, the physical category contained the most reports of cues, followed by emotional, then cognitive. This distribution of reported cues could be indicative of the relative frequency of encountering these different cues in assisted living environments. However, it may also be the case that physical cues were both salient to observe and easier to recall.
Specific Cue Categories and Subcodes
Cognitive cues included descriptions of residents appearing confused or disoriented, repeatedly forgetting things, or displaying psychotic symptoms, such as hallucinations and paranoia. For example, one participant described a resident as follows: “She used to think that people were stealing random stuff. . . . She would think that people would come in at night and take stuff.” Only 12 participants described cues that were cognitive in nature, with about half provided by a single participant who discussed many more incidents and cues than any other participant.
The physical category included cues that might be considered traditional medical symptoms as well as deviations from normal resident behavior. For example, this cue category included symptoms such as resident reports of pain, skin abnormalities (e.g., rash, swelling, bruising, sweating or clammy skin), urine that had an abnormal appearance or odor, and labored breathing. Observed deviations from typical behavior included decreased appetite, wandering or escape attempts, lethargy or lack of energy/activity, and sleep disturbances. Within this category, several cues were mentioned only once; these cues were grouped into the other category and included an array of cues that could not be grouped into any higher-level category, such as vomiting, bleeding, dizziness, seizure, and bloating. Almost all of the participants reported a physical cue (28 out of the 32 participants). The distribution of physical cues was more evenly distributed among participants; no participants described more than 10 physical cues.
Cues categorized as emotional were behaviors that were affective or attitudinal in nature. For example, one participant described a resident as both combative and noncompliant: “He’s combative, wanna fights you, wanna do nothing you ask him to do.” Participants also described residents as becoming unengaged, withdrawing from activities and interaction with other people, such as the following: “She just started to want to stay in her room all the time.” Other cues were that a resident’s general attitude or personality changed, often becoming more subdued: “She was really friendly to people and she just stopped.” Twenty participants provided the emotional cues, with one participant responsible for a quarter of those cues (the same participant described previously).
Scenario-Based Interview Results
An understanding of the nature of concerns experienced and the explanations generated by participants can be gleaned by examining the scenario-based interview. Participants’ concerns in response to the scenarios were investigated by a high-level assessment of whether concern was present or not, and if concern was present, it was followed by more in-depth inquiry into the nature of participants’ concerns (i.e., what were their explanations for what was depicted in the scenarios) as well as how they would respond to the concern.
Segmentation and coding scheme development
After transcribing the audio recordings, we segmented the transcribed data into meaningful units. A segment was defined as any utterance by participants describing (a) their explanation of why the situation presented in each scenario might be occurring or (b) an action taken to handle the scenario in question. A coding scheme was developed to categorize the segments. Within each of the categories, subcodes were developed using both a data-driven approach and the existing literature.
After the coding scheme was iteratively developed and relatively complete, intercoder agreement (degree of consistency) was calculated. A high level of intercoder agreement ensured that the coding scheme was valid and well defined and was not limited to use by the individual who created it. To measure intercoder agreement, a transcript was selected and coded independently by two coders (the first author and a trained research assistant) using MAXQDA, a qualitative data analysis software package. Percentage agreement was calculated between the two coders, and discrepancies were discussed, resulting in revisions to the coding scheme. This process was repeated a second time on a new transcript, at which point 81.2% intercoder agreement was reached. Although there is no standard, Saldana (2012) reported that agreement between 80% and 90% seems a minimal benchmark. Therefore, at this point, the remaining discrepancies were again discussed and the coding scheme revised, resulting in the final coding scheme that was used for the remainder of the transcripts. Each remaining transcript was divided up between the two coders such that the first author coded the four to five of the scenarios and the second coder coded the remaining two to three scenarios. This method ensured that each coder was exposed to every transcript and also coded different scenarios for different participants.
Presence of concern
The first interview question posed after presentation of each scenario asked participants to indicate whether the scenario would cause them concern about the resident. Participants’ responses were coded as yes/depends or no. Yes/depends included responses in which the participant responded in the affirmative as well as responses in which the participants described conditions that, if satisfied, would lead to concern. For example, for the scenario in which a female resident has wet her pants, participants indicated they would be concerned if the resident was not someone who typically had incontinence problems or if the resident had repeated instances of pants wetting rather than an isolated incident. If participants responded with no, they were asked to explain why they were not concerned. In these instances, the majority of the responses were either that (a) the situation could be resolved easily or (b) the situation was a function of aging and therefore common among residents.
Responses to the presence-of-concern interview question for each of the seven scenarios are in Table 7. The trouble walking–dizziness scenario elicited concern from all participants. As expected, the dinner-complaint scenario evoked the least concern, although almost half of participants did still express concern. For the other scenarios, the vast majority of participants did express concern. However, the incontinence and forgotten-conversation scenarios had five and nine participants (respectively) who reported no concern. It may be that the cues presented in these scenarios are associated with aging and therefore not viewed as acute problems that require any type of action or treatment. However, such an inference may be an incorrect interpretation in certain cases and will be discussed in more depth with respect to the incontinence scenario.
Presence of Concern by Scenario
Nature of the concern
Participants were asked to provide possible explanations for what was causing the behaviors and symptoms presented in each scenario. Participants could provide as many possible explanations as they wished. These explanations were coded as general, specific, or don’t know. General explanations included that the resident is sick, his or her general health is declining, or general attributions to aging. Specific explanations included references to cognitive/emotional/social issues (e.g., Alzheimer’s disease, depression), physical health issues (e.g., diabetes, stroke, urinary tract infection), or the resident’s preference not being met (e.g., does not eat because he or she does not like the food).
The explanation code frequencies are presented in Table 8. A chi-square goodness-of-fit test was performed to determine whether the distribution of explanations collapsed across all seven scenarios was equal among general, specific, or don’t know. The distribution was not equal, with fewer general explanations (14%) and don’t know (1%) responses but more specific explanations (85%), χ2(2, N = 457) = 564.15, p < .001. This same pattern was found for each individual scenario, although the ratio of general to specific explanations varied across scenarios. These data highlight that participants’ explanations tended to be specific rather than general. Within the specific subcodes, explanations were not equally distributed among the categories, χ2(3, N = 389) = 257.82, p < .001. Rather, there were more coded as physical health issue (58%) and fewer coded as resident preference not met (5%) or other (12%).
Explanation Code Frequency by Scenario
p < .05 (i.e., high-level category is significantly different from an equal distribution).
p < .05 (i.e., subcategory is significantly different from an equal distribution).
Notably, only three participant responses were coded as don’t know across all seven scenarios, and these three responses were made by three different participants. Two of these responses were made in response to the crossword trouble–confused speech scenario and one in response to the incontinence scenario. Although this finding represents a very small fraction of the total responses, these are cases where a patient’s well-being could be at risk. By not having an idea of what may be causing the symptoms, workers may fail to take the needed action.
The variability in explanations given across participants was also examined. The total number of explanations aggregated across all seven scenarios for each participant ranged from one to 21. Most participants (44%) provided fewer than 10 explanations, 37% of participants provided between 10 and 15 explanations, and 19% of participants offered between 16 and 21 explanations for the various scenarios.
Due to space limitations, only three of the scenarios will be discussed in greater detail in this paper. (The full analysis of the scenario-based interview data can be found in McBride, 2014). This subset was chosen because it included two scenarios with physical cues and one scenario with cognitive cues. These scenarios also illustrate the variability of participant responses; some scenarios elicited many interpretations, whereas others result in a more limited set of interpretations.
Incontinence
Explanations for the incontinence scenario were primarily related to physical health issues (see Figure 3). The majority of comments described the individual as being incontinent, which did not actually explain why the episode of incontinence occurred but was just a restatement of the symptoms described in the scenario. For instance, one participant described, “It could be that she’s just becoming incontinent and doesn’t know [that she wet herself] because that happens a lot in geriatrics.” In other cases, participants pointed to bladder issues, such as weakening of the bladder (e.g., “Her bladder isn’t all that strong”). Although these aging-related changes may often be the cause, incontinence can also result from an underlying issue that can be treated. Along these lines, six participants mentioned the possibility of the resident having a urinary tract infection, and four mentioned the incontinence could be related to medication. Given that these are common, treatable causes of temporary incontinence, it was surprising that they were not mentioned by more participants. This finding suggests that many participants may not recognize these potential causes and instead treat incontinence as a normal consequence of aging, possibly missing the opportunity to intervene.
Frequency of explanations coded as physical health issue for the incontinence scenario.
Although not the most frequently mentioned category, explanations coded as cognitive/emotional/social issues were discussed by several participants. These explanations described that the episode of incontinence might have resulted from a resident feeling anxious, depressed, or even afraid, potentially as a result of abuse. Some of the explanations in this category also noted that incontinence may be related to Alzheimer’s and dementia, in that the task of going to the restroom is one of the things that are often forgotten (e.g., “She could be starting to get dementia . . . forgetting to go to the bathroom”).
Trouble walking–dizziness
With respect to the scenario depicting a resident who appears dizzy and is having trouble walking, participants’ explanations were almost entirely related to a physical health issue (see Figure 4). However, even within this category, participants discussed a very wide range of potential issues. The most commonly mentioned explanation was that the dizziness and trouble walking were potentially related to medication, in the form of either a change in medication, new medication, or a missed dose (e.g., “It could be overmedicated or under”). The other category contained a large number of explanations; however, there was little consistency among comments in this category. Included were mentions of too much exposure to the heat, being bothered by legs or feet, vertigo, cholesterol, low vision, ear problems, alcohol, heart issue, and a sinus infection. The next most discussed explanations were hypertension/blood pressure (e.g., “My first thought would be that it would be a blood pressure issue”) and lack of food and/or water (e.g., “She could be dehydrated, she could be really hungry”). The remaining explanations were mentioned by fewer than 10 participants and included diabetes/blood sugar, injury, stroke, getting up too quickly, arthritis, heart attack, allergy, fatigue, cold/flu, and urinary tract infection. Participants clearly demonstrated a high level of variability in terms of what issues might be responsible for the behavior depicted in the scenario. Also interesting to note was that the explanations mentioned ranged widely in terms of criticality. For instance, there was mention of a possible heart attack or stroke (highly critical) as well as getting up too fast or being fatigued (less critical).
Frequency of participant explanations coded as physical health issue for the trouble walking–dizziness scenario.
Crossword trouble–confused speech
This scenario was the only one that elicited a comparable number of explanations from the cognitive/emotional/social issue and physical health issue categories. Explanations coded as cognitive/emotional/social issue were predominantly related to Alzheimer’s/dementia and the resident’s memory getting worse (e.g., “This might be an early sign of dementia”), although the scenario did not explicitly contain any reference to memory problems.
There was a greater variety of explanations mentioned in relation to a physical health issue (see Figure 5). Medication-related issues, stroke, and urinary tract infection were the three top-discussed explanations. The remaining physical health issues spanned fatigue, diabetes/blood sugar, hypertension/blood pressure, cold/flu, heart attack, and pneumonia.
Frequency of participant explanations coded as physical health issue for the crossword trouble–confused speech scenario.
Response to the concern
In addition to providing explanations for why the situation depicted in the scenario might be occurring, participants were also asked to describe what actions they might take to handle the situation. The range of actions described by participants is presented in Table 9. The first category of actions, gather/use information, referred to participants obtaining information or using information they may have already known about the resident to assist in assessing the situation. Several of the other categories of actions described involving or communicating with other health care professionals either within or outside of the facility. For example, participants’ descriptions of going to the medication technician, nurse, or supervisor for assistance were coded as report to staff with higher authority, whereas notifying the other caregivers of the situation was coded as inform other staff.
Participants’ Reported Actions Collapsed Across All Scenarios
p < .05 (i.e., category is significantly different from an equal distribution).
Participants also discussed taking steps to address a resident’s immediate needs, such as sitting the resident down to avoid a fall, cleaning the resident up and getting him or her into clean clothes after an episode of incontinence, or offering a substitute meal if the resident did not want to eat the offered entrée. Other actions included notifying the resident’s family, providing encouragement and assistance, redirecting or reorienting the resident if he or she was confused, making a long-term change (e.g., switching from underpants to adult diapers, using a walker or wheelchair), monitoring the resident, and documenting the situation in facility log books.
A chi-square goodness-of-fit test showed that actions were not reported with equal frequency, χ2(12, N = 1328) = 4442.18, p < .001 (see Table 9). There were more instances of gather/use information (55%) and report to staff with higher authority (10%) and significantly fewer for the remaining action categories except for address immediate need (each category <6%). Gather/use information was consistently the most discussed action across all scenarios. Second most discussed was report to staff with higher authority except for dinner-complaint and trouble walking–dizziness scenarios, in which it was address immediate need.
Because gather/use information was by far the most frequently discussed action, it will be examined in greater detail. The reason that this category combines two related but separate actions is that a clear distinction between gathering and using information was not possible due to the manner in which participants described these actions. Often participants stated that to understand the situation presented in the scenario, “it would depend on” or they “would need to know” various pieces of information, such as the resident’s health status. Because the discussion was based on scenarios of fictional residents, participants could not express whether the information in question was something they already knew or would have to acquire.
Within the gather/use-information category, comments were further coded in terms of what type of information was mentioned, including cue elaboration, resident characteristics and history, resident current and recent state, resident health, and resident family dynamics. The cue-elaboration category was used to capture participants’ requests for more specific information regarding cues presented in the scenario, such as duration, frequency, or severity of the cue. For example, in the cough–confusion scenario, participants discussed wanting to find out how long the cough and confusion had been occurring, what type of cough the resident had, and whether the resident was confused about other things besides his or her location. Participants also discussed wanting information about the resident’s characteristics and history (e.g., routine, likes/dislikes), current and recent state (e.g., how the resident is feeling, what the resident was doing recently), health (e.g., existing conditions, symptoms), and family dynamics (e.g., how often family visits).
Within several of the gather/use-information categories, participants’ reported actions were coded in terms of the level of specificity used. For example, gathering information about a resident’s health could be general (e.g., does the resident feel sick) or specific (e.g., what are the resident’s vitals). A chi-square goodness-of-fit test revealed an unequal distribution, with more comments related to specific information (80%) and fewer requests for general information (20%), χ2(1, N = 566 = 199.46), p < .001.
Summary
The data from the critical incident interview revealed a sampling of the types of cues that direct care workers have encountered and interpreted as reason for concern about residents’ health or well-being. The majority of cues discussed were specific in nature rather than general or gist recollections that something with the resident was off or unusual. Among the specific cues discussed, the majority was classified as physical, followed in frequency by cues classified as emotional, then cognitive. The diversity of cues used by participants reveals that these direct care workers were attuned to many aspects of their residents, including behavior, appearance, emotional expression, and cognitive status.
In the scenario-based interview, participants expressed concern for the majority of the scenarios presented to them. Participants’ explanations for the scenarios were typically specific in nature rather than general, and participants generated a wide range of explanations across the various scenarios, with the majority classified as physical health issues. Scenarios that included physical cues typically received predominantly physical explanations, whereas scenarios with cognitive cues tended to receive mostly cognitive/emotional/social explanations. For several scenarios, there was a wide variety of explanations that were provided rather than just a small, consistent set of interpretations. Last, participants described a wide range of actions that might be relevant across the various scenarios. Participants’ most frequently discussed course of action to handle concerns was gathering and using information, such as background about the resident or more details about the cues that initiated their concern.
Discussion
The goal of this research was to gain a greater understanding of decision making among direct care workers in assisted living, who are the eyes and ears of resident care. Specifically, this study aimed to explore the components of decision making, including cues, explanations, and actions. The various cues participants described as initiating concern in the critical incidents that were categorized as cognitive, physical, or emotional. Many of the cues discussed by participants in the critical incident interview mirror Tingström et al.’s (2010) findings on the early signs and symptoms of infections detected by nursing assistants. Although the categorization of cues differed, many of the cues reported by the present study’s participants were also reported by that sample of nursing assistants working in nursing homes. Participants in Tingström et al.’s study were prompted to focus specifically on indicators of infection, whereas the present study’s participants were given no such restriction in terms of what types of incidents to consider. That many of the cues reported are consistent between the two studies suggests that these cues are likely indicators of multiple health issues rather than just infection. Tingström et al. did not provide any frequency data regarding how often each indicator was reported; therefore it is not possible to determine whether the distribution of cues among the cognitive, emotional, and physical categories was consistent across studies.
Participants were able to provide explanations for the situations presented in the scenario-based interview, and these explanations varied across the different scenarios. The explanations were predominantly specific in nature, although general explanations, such as “He [she] may be sick,” were provided to a lesser extent. Specific explanations were primarily related to physical health issues except for the crossword trouble–confused speech scenario, in which cognitive/emotional/social issues were discussed to a greater extent. These data provide compelling evidence that this group of caregivers, who may often be undervalued in the caregiving hierarchy, is quite knowledgeable and sensitive to the issues that might affect residents.
The vast majority of actions participants reported taking in response to the scenarios involved gathering or using information. Although this component of the decision-making process is not new (Carroll & Johnson, 1990), Sund-Levander and Tingström’s (2013) focus was on the strategies that were used to gather and evaluate information rather than on the nature of the information itself. The data from the current study speak to the categories of information that direct care workers discussed as being relevant to their decision-making process, which included information about the resident’s health, current and recent state, personal history, and family. Participants also discussed the need to gather information related to the specific cues that had been observed, such as duration, frequency, and severity.
Although gathering and using information was the most frequently discussed action, participants also reported a range of other actions, such as reporting the scenario to someone with higher authority, involving outside health care professionals and emergency services, notifying family, encouraging and comforting residents, monitoring, and documenting. This finding greatly adds to the choice of actions discussed by Sund-Levander and Tingström (2013), which was essentially just reporting it up the chain of command or not. The current study’s data serve as one of the few, perhaps only, accounts of the variety of actions that direct care workers in assisted living report engaging in. These data suggest that direct care workers may be doing much more than previously realized when faced with concerning situations. Their investigative abilities are an important asset to the caregiving process.
Practical Contributions
One outcome of this study is a more detailed profile of direct care workers in assisted living settings. There is a tendency to think of these direct care workers as performing a simple job, perhaps as a result of the low pay and typically lower levels of education that is characteristic of this group of professionals. However, the data described here provide evidence that most direct care workers are engaging in relatively complex cognitive processes. They are not simply observing and reporting, but they are investigating why residents behave in certain ways or experience different symptoms and then responding to these issues with a variety of approaches. They expressed how so much of their caregiving behavior depends on numerous factors, such as the intricacies of each individual resident, and that they have to be cognizant of these details and use them in their everyday work.
In addition to a better understanding of the complexity of caregiving, the findings from this study can contribute to the delivery of care for the older adult population in numerous ways. The range of cues identified may serve as guidelines that can be used in the training of assisted living direct care workers. The findings of this study highlight that important cues may present as cognitive, physical, or emotional changes in residents. The cues identified by the present study can be provided as concrete examples of the variety of cues that might signal a health or well-being issue. One particularly challenging aspect of detecting health issues is that many of the typical symptoms and cues are not present in geriatric populations (Rehman & Qazi, 2013). Therefore, there is likely a need to educate direct care workers who are working in long-term-care settings about the unique characteristics of older adults.
As an example of how cues might be put into action, Tingström et al. (2015) developed an early-detection-of-infection scale that allowed nursing home assistants to formally document changes in specific signs and symptoms of infection. The researchers were then able to assess the validity of nursing assistants’ interpretations by having two physicians judge each episode in which a nursing assistant documented a suspected infection using the instrument. A model of the instrument was then constructed that correctly identified patients with infection in 84% of cases.
Training of direct care workers may also be supplemented by utilizing similar interview methods as those used in this study. A form of continuing education might include presenting workers with scenarios, asking how workers would handle those scenarios, and then correcting or supplementing their responses with any additional tasks that should be carried out or cues that should be attended to. Sharing relevant incidents among caregiving staff would also afford another opportunity to correct any misconceptions and emphasize how important these workers are to keeping patients well.
Additionally, the cues identified in this study may serve an audience beyond assisted living. Caregivers in home health might greatly benefit from training on potential cues, particularly given that they are often delivering care without any peers or immediate access to a nurse or supervisor. In these situations, an appreciation of how seemingly innocuous cues might indicate serious health issues is critical to maintaining an older adult’s health and well-being. This information may also be useful to informal caregivers, such as family members or friends of older adults, in addition to professional caregivers. Informal caregivers are likely already engaging in the task of detecting changes in a loved one, but having little or no training puts them at a disadvantage at interpreting and responding to critical cues. These individuals would also potentially benefit from guidance regarding how to appreciate and interpret various cues.
Another implication of the current research is identification of direct care workers’ information needs. Although Sund-Levander and Tingström (2013) described the strategies used to gather and evaluate information, the nature of the information used by caregivers was unknown. By asking participants what information they would want during the scenario-based interview, participants could think about ideal information gathering. The information they discussed may not be readily accessible if communication practices are poorly designed or are not documented in patient logs or incident reports. Identifying information needs may be used to train direct care workers on the necessity of documentation and communication, as well as revealing opportunities for data collection and data sharing technologies.
Future Directions
Many opportunities exist to build upon the current research. The critical incident approach utilized self-reporting of cues, which is heavily reliant on participants’ memory for events. Although it is tempting to assert that the relative frequency of the cues mentioned may be indicative of their actual occurrence, doing so would be inappropriate. Because participants chose what incidents to discuss with the interviewer, their decision to describe one type of incident over another may have been driven by multiple factors, including the recency of the incident, the uniqueness of the cue, or the severity of the issue or outcome. Exploring the reasons a particular set of incidents and/or cues comes to mind first would be informative. Future research might also inquire as to whether the likelihood of a cue being detected is equal to its rate of occurrence or whether certain cues are more likely to be ignored or missed. Identifying those missed opportunities would help improve training of direct care workers.
The drawbacks of self-report data were also present in the scenario-based interview. When discussing what information participants would need or use, they may have described information or knowledge that they might not actually have or be permitted access to in their work environment. For example, participants described that they would need to know about the medication the resident was taking. One direct care worker might actually already have this information stored in their mind, whereas a different direct care worker might need to examine resident’s records or confer with another staff member to discover this information.
Alternative approaches to studying decision-making processes among direct care workers that are not subject to the limitations of self-report include observational studies or analysis of incident documentation. Observing staff as they work with residents might give a more accurate depiction of the frequency with which different issues present themselves as well as how they respond, including what information is gathered or used. Examining reports of incidents that have already occurred may be another valuable avenue of inquiry. This information often resides in a log book that nursing staff use to document any changes in resident status as well as more formalized incident reports completed after more serious incidents occur, such as falls.
The current study included direct care workers from 19 different assisted living facilities. Assisted living facilities vary in many ways, including the number of residents, staff hierarchy, management practices, and so on. These factors could have influenced how participants responded to the scenarios. Therefore future endeavors may benefit from controlling for these facility dimensions or by purposely recruiting to allow comparisons based on facility characteristics.
Another limitation of the current study was the difficulty in judging the accuracy of participant responses in the scenario-based interview. As a general check on accuracy, a subject matter expert was asked to independently evaluate the scenarios and provide her opinion of the most likely causes. This evaluation was compared to participant responses, and for all of the scenarios presented here, there was considerable agreement. However, future work might benefit from using scenarios based on actual incidents in which the root cause of the symptoms or behavior is known and can be used to determine how accurately caregivers assess and respond to the scenario. Doing so would necessitate determining the prevalence of different health issues among older adults as well as the cue validity of various symptoms displayed by older adults with those health conditions. Also of value would be to apply Brunswik’s lens model framework (Hammond & Stewart, 2001) to determine the association of cues with particular outcomes. With that information, it would be possible to judge how direct care workers are utilizing the cues presented in a given situation.
Last, having accurate knowledge of what constitutes normal aging versus a treatable, medical condition is critical for direct care workers in assisted living. However, no assessment instrument exists to evaluate direct care workers’ knowledge of aging. The knowledge-of-aging questionnaire that was developed for the purposes of this study had low reliability, meaning the results could not be interpreted. Such an assessment tool would be a valuable component of training and would allow administrators or managers to check whether their staff have the appropriate understanding of typical age-related changes.
Advancing NDM
This study represents an important addition to the field of NDM by examining a workforce and domain that has not previously been the subject of study in the area. Direct care workers may have relatively lower levels of training and education, but the job they do is critical and complex. They are on the front lines of providing care to older adults in assisted living, who depend on them to notice warning signs of health and well-being issues, some of which may life-threatening. This study also demonstrated that traditional NDM methods, such as a critical incident interview and scenario-based interview, can be successfully used with this population. The knowledge gleaned from this study has a clear applied value for training and for the design of information support systems.
As research continues in this area, understanding of how direct care workers in assisted living and other fields of long-term care make decisions will continue to grow. As older adults seek out these forms of care, it will become even more critical to share these insights with the caregivers and administrators in long-term care so they can continue to improve the delivery of care and support the aging population.
Footnotes
Acknowledgements
This research was supported in part by a grant from the National Institutes of Health (National Institute on Aging) Grant P01 AG17211 under the auspices of the Center for Research and Education on Aging and Technology Enhancement (CREATE; 
). Some data and analyses presented here were part of the first author’s doctoral dissertation (Bowman née McBride, 2014). Portions of these data were presented at the Human Factors and Ergonomics Society’s 58th Annual Meeting (Chicago, Illinois, October 2014) and appear in the proceedings of that conference (Bowman & Rogers, 2014). Special thanks to Sean McGlynn and Evelyn Chang for their assistance during data analysis and coding.
Sara E. Bowman (née McBride) is a visiting lecturer in psychology at Georgia State University. She received her PhD from the Engineering Psychology program at the Georgia Institute of Technology.
Wendy A. Rogers is a professor of psychology at the Georgia Institute of Technology. Her research interests include design for aging, technology acceptance, human–automation interaction, aging-in-place, human–robot interaction, cognitive aging, aging with disabilities, and skill acquisition and training. She is the director of the Human Factors and Aging Laboratory () and a Certified Human Factors Professional (BCPE Certificate No. 1539).