Abstract
The population of older adults in the United States is increasing rapidly, with approximately 77% of this population experiencing one chronic condition and 50% experiencing two or more (National Center for Chronic Disease Prevention and Health Promotion, 2011). Medications are often the first intervention used to treat chronic conditions, extend life expectancy, and improve quality of life (Unni et al., 2013). As a result, nearly 90% of all Medicare beneficiaries take medication, and half of the older adults who take medication take five or more (Brown & Bussell, 2011; Qato et al., 2016). Medications, when taken correctly, have the potential to be one of the most effective and affordable interventions to manage health conditions and treat disease (Wilson et al., 1996). However, 40% to 75% of community-dwelling older adults are not able to correctly adhere to their medication routine (Brown & Bussell, 2011; Wolf et al., 2011). Consequences of nonadherence can be very serious and include decreases in medication effectiveness and increases in adverse events, medication side effects, institutionalization, and even death (Weng et al., 2013). Reasons for nonadherence vary significantly and can include declining to initially fill a prescription, forgetfulness, lack of information, or not taking a medication as prescribed (Brown & Bussell, 2011; Jimmy & Jose, 2011; Qato et al., 2016). Correctly identifying the reasons for nonadherence and the barriers contributing to it is essential to improving adherence outcomes and can be done through a medication management assessment.
Although many medication management assessments exist, the majority are limited in ability to identify the actual barriers to medication adherence. Some assessments use only standardized pills and instructions (e.g., the Hopkins Medication Schedule; Carlson et al., 2005), which limits the ability to identify any specific medication management problems faced by the older adult, and others are screening measures or checklists (e.g., the Medication Management Instrument for Deficiencies in the Elderly [MedMaIDE]; Orwig et al., 2006) that do not identify the underlying reason for difficulties in medication management. Although these assessments may be used effectively in specific situations (e.g., discharge from inpatient rehabilitation), they do not have the ability to measure person–environment fit, that is, the match of an individual’s abilities with the demands, or barriers, of the environment (Nahemow & Lawton, 1967). Measuring person–environment fit during the medication management process is crucial in determining how a person’s ability to manage medication is influenced by the barriers in the home environment. The home setting is the most reliable milieu in which to assess medication management ability because it is where medication management takes place for the majority of older adults (Farris & Phillips, 2008).
To fill this gap, we developed the In-Home Medication Management Performance Evaluation (Home–Rx), a performance-based medication management assessment to be used in the homes of community-dwelling older adults to assess ability to manage medication routines in context, identify risk factors for medication management problems, and identify the environmental barriers influencing medication management ability. The HOME–Rx consists of three sections: Part 1, the Medication Management Risk Factor Interview; Part 2, the Medication List; and Part 3, the Medication Management Assessment. The Medication Management Risk Factor Interview includes 21 questions that assess an older adult’s risk factors in regard to medication mismanagement (e.g., multiple prescribing physicians, multiple filling pharmacies). The Medication List compiles information on the name, dosage, time of day, frequency, special instructions, and any additional important information for each prescription and over-the-counter medication and for vitamins and supplements. The Medication Management Assessment includes 11 performance-based medication management tasks that are conducted using the older adult’s prescriptions; these tasks include storing and retrieving pills; reading labels; verbalizing the dosage instructions, special instructions, and purpose; following dosing directions correctly and recognizing when one has missed doses; opening containers; setting up medications; taking out medications; and physically administering medications. The content validity has been established through a rigorous process, and the results of that study are reported elsewhere (Murphy et al., 2017). In brief, content experts (n = 7) agreed that the overall instrument was valid for measuring medication management (scale-level content validity index = .95) but noted that it needed further development, including more sensitive and thorough scoring scales and the establishment of additional psychometric properties.
The purpose of this study was to further develop the HOME–Rx by modifying the scoring metrics, improving its clinical utility, and establishing additional psychometric properties. We hypothesized that the changes to the HOME–Rx would result in a clinically useful, psychometrically sound instrument to be used to measure medication management ability among community-dwelling older adults.
Phase 1: Modifying the Scoring Metrics and Improving Clinical Utility
Method
This study was completed in two phases. The institutional review board at Washington University in St. Louis approved all study procedures, and all participants provided informed consent. In Phase 1, the scoring metrics were modified and the clinical procedures were evaluated.
Participants.
Older adults in the St. Louis, Missouri, area who were prescribed three or more medications, managed their own medication routine, and lived in their own home were eligible to participate. Those with cognitive impairment indicated by a score greater than 10 on the Short Blessed Test (Katzman et al., 1983) were ineligible. To identify participants for Phase 1, occupational therapy practitioners at Washington University identified a convenience sample of participants from the research laboratory’s community advisory board.
Procedure for Modifying the Scoring Metrics.
As originally designed, performance of the 11 medication management tasks in Part 3 of the HOME–Rx was scored on a 3-point scale, and performance was the only score recorded during that part. Feedback from content experts during the content validation process indicated that the scoring of the Performance subscale needed to be expanded to be more sensitive to change in performance in medication management abilities and to provide meaningful scores to guide the intervention process. Furthermore, the experts suggested adding measures of capacity, safety, and impact of environmental barriers to identify the unique aspects of the home environment that influence the older adult’s ability to successfully manage medication. The Capacity, Safety, and Barriers subscales were modeled after already-existing, valid assessments: the In-Home Occupational Performance Evaluation (I–HOPE; Stark et al., 2010), the Performance Assessment of Self-Care Skills (PASS; Chisholm et al., 2014), and the I–HOPE for Providing Assistance (I–HOPE Assist; Keglovits et al., 2015). The scoring metrics from these assessments were chosen because they are established processes for reliably providing objective scores of performance, capacity, safety, and severity of environmental barriers for activities performed in the home.
The Performance subscale of the HOME–Rx measures the older adult’s current ability to complete specific medication management tasks, whereas the Capacity subscale measures the potential ability of the older adult to complete the task if proper supports are in place (e.g., increased font size on labels, presorted blister packs). Both are scored on a 5-point Likert scale, with 0 indicating inability to complete the task and 4 indicating independence. The Safety subscale measures the older adult’s ability to safely complete the task and is scored on a 4-point scale with 0 indicating that the task had to be stopped because of significant safety concerns and 3 indicating no concerns during the task.
The Barriers subscale is used to identify and score environmental barriers, or characteristics of the environment that impede performance. On the basis of observations of the older adult completing each of the 11 medication management tasks in Part 3, the occupational therapist identifies the unique aspects of the environment that decrease performance. If an older adult has difficulty sorting a medication because the pills are very small, the lighting is dim, and there is no clear space for sorting, the environmental barriers are identified as “small size of pills,” “lack of adequate lighting,” and “lack of clear counter space.” Each task in Part 3 can have multiple barriers, and most often do. After each barrier is identified, the severity of the barrier for each task is scored on a 6-point Likert scale: 0 (no impact on performance), 1 (standby assistance required), 2 (minimal assistance required), 3 (moderate assistance required), 4 (maximum assistance required), or 5 (completely unable to complete the task). For example, if the small size of the pills means that the older adult cannot pick up pills at all, that barrier would receive a score of 5.
Procedure for Improving Clinical Utility.
To ensure that the HOME–Rx has good clinical utility, meaning it is easy to administer and score (Anemaet & Moffa-Trotter, 1999), we modified the administration procedures and the formatting of the assessment. The assessment took an average of 65 to 75 min to complete, which limited its utility. Revisions to procedures included changing the administration order of the three parts to allow for better flow, less redundancy, and more time for clinicians to build rapport. In addition, we changed the order of the performance tasks within the Medication Management Assessment to better match an older adult’s medication management routine. We also reformatted the layout of the assessment to allow for easier scoring and standardized the administration instructions.
Finally, it became apparent that the majority of older adults in our sample did not have a current medication list and were unable to independently generate a complete and accurate list during Part 2. In response, we added a pictorial tool, Medication Reconciliation–Review of Systems Subject (MR ROSS), to increase recall of medication by older adults (Vouri & Marcum, 2013). MR ROSS includes a basic picture of a human being with primary body systems labeled. It provides a systematic method for health care professionals to accurately generate a medication list for older adults by asking them to correctly identify medication taken for each body system (Vouri & Marcum, 2013). Adding MR ROSS to the HOME–Rx decreased the need for older adults to use rote memory, which is often not reliable, to recall medication (FitzGerald, 2009; Gizzi et al., 2010; Vouri & Marcum, 2013).
Results
Revisions to the HOME–Rx were pilot tested by two occupational therapy practitioners in the homes of four community-dwelling older adults who had a mean age of 73.8 yr, 50% of whom were female, and the majority of whom were White (75%). They took an average of 9.7 prescription medications. The assessment was administered until a consensus on the scoring and formatting was reached by all members of the research team. The scoring scales were established, and because of the changes to the formatting of the HOME–Rx, administration time was reduced from an average of 65 to 75 min to 25 to 35 min. Finally, we evaluated the effectiveness of the MR ROSS tool in helping therapists identify missing medications. All four participants omitted medications when asked to create a list, but after using MR ROSS the therapists identified a mean of 2.5 additional medications for each participant.
Phase 2: Establishing Concurrent Validity and Interrater Reliability
Method
Participants.
In Phase 2, we recruited 30 older adults to determine the validity and interrater reliability of the HOME–Rx. A variety of recruitment methods were used, including contact with previous research study participants, flyers distributed at exercise groups for older adults, and word of mouth. The sample size of 30 for Phase 2 was based on the sample size for interrater reliability studies of similar assessments (Berg et al., 2004; Keglovits et al., 2015).
Measures.
Two occupational therapy practitioners completed the measures in participants’ homes. Demographic information, including age, education, gender, marital status, ethnicity, living situation, and health care services received, was collected. The number and impact of comorbid conditions were recorded using the Charlson Comorbidity Index (Charlson et al., 1987). In addition, each older adult completed the HOME–Rx, the MedMaIDE, and the Medication Management subscale of the PASS, a performance-based, criterion-referenced, and client-centered observational tool that measures the older adult’s ability to live independently and safely in the community by assessing performance on various activities of daily living and instrumental activities of daily living (Chisholm et al., 2014). The MedMaIDE covers three domains important for ensuring medication compliance (i.e., knowledge of medications, how to take medications, and procurement of medications). Responses are scored “yes/1 or no/0,” yielding a total deficiency score, with a maximum score of 13 indicating the highest level of deficiency (Orwig et al., 2006). Independence, adequacy, and safety are rated on 4-point scales.
Procedure.
Three occupational therapy practitioners (Emily Somerville, Kayla Massey, and Marian Keglovits) completed a 4-hr training session that included an introduction to the HOME–Rx, a review of the assessment, an explanation of the scoring procedures, a demonstration, and practice administering the assessment. Two raters completed all assessments for each participant; one rater was randomly (using a software application) chosen to conduct the assessment while the other rater observed and scored the HOME–Rx, MedMaIDE, and PASS independently. At the conclusion of the visit, the observing rater had the opportunity to ask clarifying questions. The raters were instructed not to discuss the assessment results among themselves. Both therapists completed the assessment at the same time, instead of at two separate times, to control for the variability that occurs in older adults’ living environment on a daily basis (Gitlin et al., 2002).
Data Analysis.
Data entry was completed using Research Electronic Data Capture (REDCap), a secure web application for building and managing online surveys and databases (Harris et al., 2009). Data analysis was completed using IBM SPSS Statistics (Version 23; IBM Corp., Armonk, NY). Demographic characteristics of the participants were calculated using descriptive statistics. Correlations between the total MedMaIDE score and the Performance and Barriers subscales of the HOME–Rx, as well as correlations between the Performance and Safety scores of the Medication Management subscale of the PASS and the Performance and Safety subscales of the HOME–Rx, were computed to determine the concurrent validity of the HOME–Rx. All correlations were calculated using Pearson’s correlation coefficient (Portney & Watkins, 2009).
Interrater reliability was determined by calculating intraclass correlation coefficients (ICCs) comparing scores between the trained raters. Shrout and Fleiss’s (1979) Model 2 was used, which assumes that all participants are assessed by the same raters, who are considered representative of a larger population. According to Shrout and Fleiss’s criteria, agreement is considered excellent when the ICC > .75, whereas ICCs <.75 indicate moderate to poor reliability.
Results
Descriptive statistics for the participants in Phase 2 are provided in Table 1. We addressed concurrent validity by examining correlations among the HOME–Rx, PASS, and MedMaIDE. We hypothesized that the Performance and Safety subscales of the Medication Management Assessment section in the HOME–Rx would be positively correlated with the Performance and Safety scores from the Medication Management subscale of the PASS and that the Performance subscale would be negatively correlated with the MedMaIDE. In addition, we hypothesized that the Barriers subscale of the HOME–Rx would be positively correlated with the MedMaIDE. The PASS was positively correlated with the HOME–Rx Performance subscale (r = .57, p < .001) and Safety subscale (r = .49, p < .001). The MedMaIDE was negatively correlated with the HOME–Rx Performance subscale (r = −.69, p < .001). The MedMaIDE was positively correlated with the HOME–Rx Barriers subscale (r = .70, p < .001).
Participant Characteristics
Note. N = 30. M = mean; SD = standard deviation.
We calculated ICCs for the HOME–Rx subscales, which ranged from .87 to 1.00. The strength of agreement for the I–HOPE Assist (Keglovits et al., 2015) was excellent for all subscales (Portney & Watkins, 2009). These results are displayed in Table 2.
Interrater Reliability for the HOME–Rx Subscales
Note. HOME–Rx = In-Home Medication Management Performance Evaluation; ICC = intraclass correlation coefficient.
Discussion
The HOME–Rx is a new assessment tool with excellent validity and reliability that was designed to determine the risk factors for medication mismanagement in older adults, assess the ability of older adults to correctly and safely complete their medication management routine, and measure the severity of environmental barriers’ impact on medication management tasks. The HOME–Rx fills a gap in medication management assessments for community-dwelling older adults because it is conducted in the homes of older adults with their own medications and identifies the individual’s barriers to medication management.
In this study, we found that the HOME–Rx demonstrates convergent validity with other measures of medication management. As expected, the HOME–Rx Performance and Safety subscales were highly correlated with the Performance and Safety subscales of the PASS, indicating that the HOME–Rx is able to accurately measure both older adults’ ability to manage medications and their safety while completing medication management tasks. There was a negative correlation between the Performance subscale of the HOME–Rx and the MedMaIDE: A higher score on the MedMaIDE indicates decreased ability to perform medication management tasks, whereas a lower score on the HOME–Rx Performance subscale indicates decreased performance ability. Finally, there was a positive correlation between the Barriers subscale score on the HOME–Rx and the overall MedMaIDE score. The direction of this relationship is also as expected: An increase in difficulty in performance would likely indicate an increase in impact in environmental barriers. These correlations were all statistically significant and in the direction we hypothesized, indicating that the HOME–Rx does in fact measure the constructs of medication management performance, safety, and impact of barriers on performance.
Changes to the HOME–Rx include the addition of the Capacity, Safety, and Barriers subscales and expansion of the scoring scale for the Performance subscale from a 3-point scale to a more sensitive 5-point scale. These changes will allow the HOME–Rx to more sensitively measure change before and after an intervention and to more effectively guide occupational therapy practitioners in treatment and intervention planning. By accurately identifying and rating environmental barriers, occupational therapy practitioners can help older adults remove barriers and maximize performance through optimal person–environment fit.
The addition of MR ROSS to the HOME–Rx was valuable in that it increased the accuracy of medication recall. It is imperative that occupational therapy practitioners use a correct list of medications when implementing medication management strategies so as not to increase adherence to the wrong medication or dosing schedule.
The HOME–Rx is clinically useful. Training sessions lasted about 4 hr, and the clinicians involved in this study ranged in number of years of experience, indicating that the training session was adequate for all levels of occupational therapy practitioners to achieve reliability in rating. The HOME–Rx is administered in the homes of older adults and takes only about 25 to 35 min to complete, making it feasible to administer during an occupational therapy session. The HOME–Rx encourages a client-centered approach to treatment planning given that occupational therapy practitioners can use the results to guide an intervention tailored to addressing the specific barriers to medication management for each individual client.
Although the HOME–Rx has many strengths, it does have limitations. One potential limitation is that because it is performance based and uses the older adult’s own medication, it can be administered only in the home. The HOME–Rx is not designed to be a screening tool used by a doctor or therapist when an individual is being discharged back home (e.g., MedMaIDE). This limits the settings in which the HOME–Rx can be used; however, a performance-based, in-home assessment is always preferred (Golant, 2003) when the situation allows.
The HOME–Rx shows excellent potential for measuring change in Performance, Safety, and Barriers scores; however, the sensitivity of the subscales to detect a change in medication management is unknown. Finally, the sample for this study lacked heterogeneity. More studies in other samples with more diversity would be helpful in determining the assessment’s usefulness in a broader population.
Implications for Occupational Therapy Practice
The HOME–Rx is a valid, reliable performance-based assessment, and the results of this study have the following implications for occupational therapy practice:
The HOME–Rx provides occupational therapy clinicians and researchers with a measure of actual medication management ability in the home using the older adult’s medications.
The HOME–Rx can potentially guide occupational therapy practitioners when planning a medication management intervention by identifying specific barriers to performance, allowing the practitioners to develop a comprehensive treatment plan to eliminate barriers to successful medication management.
Conclusion
The purpose of this study was to establish the scoring metrics, clinical utility, and preliminary psychometric properties of an in-home medication management assessment to be used with community-dwelling older adults. The assessment now has expanded scoring metrics, is formatted to be administered in a timely manner, and has been shown to have good validity and reliability. Determining the performance problems of older adults with medication management is of utmost importance because nonadherence with medication is a public health concern. By using the HOME–Rx to adequately determine performance problems, capacity, safety concerns, and environmental barriers, occupational therapy practitioners can potentially use the results to guide treatment planning and improve medication management for older adults.
Footnotes
Acknowledgment
This research was completed at Washington University in St. Louis, Program in Occupational Therapy.