Abstract
Keywords
Patient decision aids (DAs) are educational tools designed to promote high-quality medical decisions. There is considerable evidence, from high-quality randomized trials, that DAs increase knowledge, reduce decisional conflict, and help patients clarify their preferences. 1 Existing DAs vary in the amount of detail, content (e.g., use of patient narratives), design features (e.g., graphics), media (e.g., use of video), and level of interactivity. The International Patient Decision Aid Standards has recommended criteria for the development of evidence-based DAs; however, the relative impact of different features on the effectiveness of specific DAs is not clear. 2
As DAs proliferate and efforts to integrate shared decision making into routine care expand, understanding the comparative effectiveness of different available tools is critical. The goal of this pilot study was to assess the feasibility of a protocol for a multisite randomized comparative effectiveness trial comparing two DAs for hip and knee osteoarthritis (OA) and gather preliminary evidence on impact of the tools.
Methods
The reporting of this pilot randomized controlled trial follows the proposed CONSORT guidelines for randomized controlled trials and SUNDAE guidelines for reporting on DA evaluation studies.3–5 Institutional review board (IRB) approval was obtained centrally through the main IRB site. All other sites ceded review to the central IRB. This study is registered on Clinicaltrials.gov (# NCT02729831).
Settings
Patients and physicians were recruited from the orthopedic departments of two sites: a large academic medical center (site 1) and a community hospital (site 2). Both sites share a common electronic medical record (EMR), which was used to track participant eligibility. Both sites also had EMR-enabled ordering of DAs and had been using Health Dialog DAs as part of routine care for patients with hip and knee OA. 6
Participants
Study coordinators screened the schedules of participating orthopedic surgeons 2 weeks prior to the visit date (pre-visit screening) to determine study eligibility of new patients (see Table 1). For many patients, there were limited data available from the scheduling system (e.g., only a note indicating reason for visit, e.g., “right knee pain,” with no prior history and no imaging confirmation of OA). These patients were assumed to be eligible, and study coordinators captured additional information during reminder calls and reviewed visit notes to confirm eligibility. One goal of the pilot was to calculate the percentage of patients who were deemed ineligible after the visit.
Criteria for Eligibility
Interventions
Decision Aid-A (DA-A): Arthritis: Should I Have Knee Replacement Surgery? (Healthwise 2015) and Arthritis: Should I Have Hip Replacement Surgery? are 15-page printed brochures. They include 7 sections (get facts, compare options, frequently asked questions, patient stories, quiz yourself, your feelings, and your decision). These DAs have been reviewed as part of the Ottawa Decision Aid Inventory and meet 7 of 7 International Patient Decision Aid Standards (IPDAS) qualifying criteria and 8 of 9 quality criteria.7,8
Decision Aid-B (DA-B): Treatment Choices for Knee Osteoarthritis (Health Dialog 2014) is a 42-minute DVD and 48-page booklet, and Treatment Choices for Hip Osteoarthritis (Health Dialog 2014) is a 44-minute DVD and 40-page booklet that include patient testimonials and detailed data on options and outcomes. These DAs are not included in the Ottawa inventory, but review by authors suggest that they meet 7 of 7 IPDAS qualifying criteria and 8 of 9 quality criteria.
Randomization and Study Design
Patients were randomly assigned to an arm using a computer-generated randomization allocation sequence. About 2 weeks before the visit, staff sent eligible participants the assigned DA, a cover letter, information sheet about the study, and a survey. The cover letter included instructions for how to opt-out. The random assignment of patients who chose to opt-out was put back into the queue and became available for the next eligible patient.
All patients received the DA by mail. The majority of patients also received the survey in the mail. Some patients at the community site who were registered users of the hospital’s patient web portal were sent the survey by email. Quality of life data were collected in the clinic by paper survey at site 1 and via iPad at site 2.
A few days before their appointment, study staff called patients to remind them to review the DA and fill out the survey. On the day of the visit, staff met patients at the clinic to collect the surveys and answer any study-related questions. If needed, study staff provided a copy of the survey for patients to review and complete in the waiting room prior to seeing the surgeon.
Physicians were not blinded to the DA intervention; however, it is unlikely that they knew which arm unless the patient brought the tool into the visit with them. Physicians did not receive any information from the patient surveys (e.g., regarding patients’ treatment preference). The study did not actively intervene in the visit and the surgeons conducted their visit according to their usual process.
Measures
Hip OA and Knee OA Decision Quality Instruments (DQI): Each DQI contains 5 decision-specific, multiple-choice knowledge items, 3 decision-specific goals and concerns (rated from 0 [not at all] to 10 [extremely] important), and one question on treatment preference. We calculated a total knowledge score (0% to 100%) and the percentage of patients who received treatments that matched their stated preference. The minimal important changes in knowledge and concordance scores are 10%. 9
EQ-5D: A 6-item summary measure of overall health status. It generates a single index value for health status on which full health is assigned a value of 1 and death a value of 0. In conjunction with weights established for the 243 different combinations, the EQ-5D can be used to obtain quality-adjusted life years and the minimum important change is 0.1 points.10–12
Decision aid usage: One item assessed how much of the DVD, booklet, and/or website was reviewed (all, most, some, none).
Single-item literacy screener: One question assessed how often patients need help reading and understanding medical paperwork. The responses were dichotomized with never versus other responses.13,14
Study staff reviewed medical records to determine surgical status (defined as having surgery within 6 months of the visit). Patients were considered to have had nonsurgical treatment otherwise.
Analyses
For our feasibility assessment, we examined opt-out rates, response rates, and rates of post-screen ineligibility. We tested whether the actual rate was different from our target of having 60% of eligible patients enroll on the study and complete the assessment before the visit.
We compared 1) use of DA, 2) knowledge scores, and 3) percentage who received preferred treatment across arms using t test for continuous variables and Pearson chi-square test for categorical variables. We examined whether literacy level affected usage and knowledge scores. We used IBM’s SPSS version 20 for all analyses.
Results
From December 2015 to March 2016, staff identified 100 potentially eligible patients who were randomly assigned to an arm. Figure 1 shows the patient flow across the study. Overall, 24 patients cancelled or did not show up to their appointment after randomization. Two patients were found to be clinically ineligible during the reminder call (for prior knee replacement and avascular necrosis) before the appointment, and thus, 26/100 (26%) were ineligible post randomization. The overall response rate for eligible patients was 78% (58/74), which is significantly higher than the target of 60% (P = 0.01). The response rates did not vary by arm, 80% (33/41) for DA-A versus 76% (25/33) for DA-B. Most sample characteristics were balanced across arms (see Table 2); only the baseline quality of life was higher in the DA-B group. The sample characteristics did not differ significantly by site.
CONSORT flow diagram of study.
Sample Characteristics
DA, decision aid.
P < 0.05.
The reported DA usage varied across the arms, with more patients reporting reviewing all of DA-A, the shorter DA (see Table 3). Combining respondents across arms, participants with low health literacy (n = 14) were as likely to review all of the DAs as those with high literacy (n = 44) (42% v. 48%, P = 0.7). Older participants (70 and older, n = 12) were less likely to review all of the DAs compared with those under 70 years old (n = 46) (44% v. 55%, P = 0.7), but the finding was not statistically significant.
Patient-Reported Use of Decision Aids (DAs)
Total knowledge scores were similar for each DA (55% DA-A v. 49% DA-B, P = 0.4). Figure 2 shows the knowledge scores by the amount of use of the DA. Patients who reviewed all the DA-B had significantly higher knowledge scores than those who did not (80%, n = 6, for those who reviewed all, v. 39%, n = 19, for those that did not review all, P = 0.004). The scores were also higher for DA-A (62%, n = 21, for those that reviewed all, v. 53%, n = 11, for those that did not, P = 0.3), but the magnitude of the difference was not statistically significant. Patients with low literacy had similar total knowledge scores to those with high literacy (52% v. 55%, P = 0.7). Older patients (≥ 70 years old) had lower knowledge scores than younger patients (55% v. 43%, P = 0.2), but the difference was not statistically significant.
Knowledge scores by usage for each decision aid (DA).
Ratings of goals and concerns were similar across arms (see Figure 3). About half of the patients in each group preferred surgery (55% DA-A v. 52% DA-B). The rest were unsure (33% DA-A v. 40% DA-B) or preferred nonsurgical treatment (12% DA-A v. 8% DA-B), P = 0.8. For the patients who were unsure, the majority went on to have surgery in the DA-A group (6/11, 55%), whereas a minority had surgery in the DA-B group (2/10, 20%), P = 0.1. For patients who stated a clear preference, a similar percentage, 77% (18/22) for DA-A and 73% (11/15) for DA-B (P = 0.8), received treatment that matched their preference. All of the mismatches (8/22) were patients who preferred surgery but did not have it within 6 months.
Comparison of importance ratings for each arm. DA, decision aid.
Discussion
This pilot study is one of the first pragmatic, comparative effectiveness studies of commercially available DAs in orthopedics. Results showed similar outcomes across these tools, despite the fact that they are very different in their content and features. The pilot provided evidence of feasibility and acceptability as few patients opted-out and the response rate exceeded the target of 60%. It also found a fairly high rate of post-randomization exclusions (26%), which is important to factor into future study recruitment estimates. The main issue in this sample was that many patients cancelled or did not show up for their appointment with the surgeon. The rate of cancellations was typical for the clinics, and staff did not feel that the study had any impact on no show rates.
Both DAs resulted in similar total knowledge scores, with the shorter DA appearing to have a slight advantage. The Cochrane systematic review of patient DAs found that more complex DAs tended to result in better knowledge scores than simpler ones. 1 Our lack of a difference may be explained by the differential usage of the short and long DAs. The shorter decision aid, DA-A, was more likely to be used in its entirety; however, those participants who reviewed all of DA-B had the highest knowledge. This finding highlights a tradeoff with existing tools where shorter ones are more likely to be used but may have more limited benefits.
The patient population seeking treatment for knee and hip OA is older and may have more issues with health literacy. In this study, the results suggest there might be a difference by age, with older patients less likely to use a DA and more likely to have lower knowledge scores. Combining respondents across arms, participants with low self-reported health literacy had similar rates of usage and similar knowledge scores as those with high literacy. Studies suggest that educational materials with less content, which focus on key findings as opposed to presenting many details, are superior for comprehension for patients with low literacy and older patients with mild cognitive impairment. 15 In this pilot, the shorter DA did not perform better in the lower literacy or the older age group; however, additional studies with larger samples are needed to examine effectiveness of the DAs based on health literacy and age with adequate power.
About half of the patients scheduled to see an orthopedic surgeon had a clear preference for surgery before the visit. The DAs did not differ in their impact on patients’ goals and treatment preferences. A significant percentage in each group were unsure (34%), and when we followed these patients over time, their final treatment appeared to differ by study arm. This pilot was too small to fully examine this finding; however, the larger comparative effectiveness study will have sufficient power to detect a difference in surgical rates across groups.
Several other studies have used the Health Dialog hip and knee OA DAs (DA-B) and have found that compared to usual care, patients were more likely to make an informed choice at the end of the first visit, 16 had higher decision quality, 17 and had lower surgical rates and overall costs. 18 Another study using DA-B randomly assigned patients to the booklet versus the booklet plus DVD and did not find a significant difference in knowledge, stage of decision making, or other outcomes; however, usage of the booklet and/or DVD was not reported. 19 Our study is the first to formally evaluate DA-A and to compare effectiveness of DAs from different vendors.
The pilot study has some limitations. As a pilot, it was a small study designed to examine feasibility and a larger study is needed to confirm findings with sufficient power. The DA order was documented in the chart, and as a result, the physicians were not blinded to the intervention. We did not survey patients after their visit, and as a result, do not know whether their preferences changed and if so how they may have changed. We only followed patients for 6 months, and it is possible that some patients had surgery scheduled outside that time frame. We collected very limited data on the patient sample, and we do not have information on race, income, employment, or insurance status. Finally, due to the small sample size we did not adjust the findings for baseline imbalances between arms or clustering of patients within surgeons.
Clinical guidelines emphasize the importance of shared decision making for hip and knee OA.20,21 However, limited evidence exists about the most effective type of DA that is able to be integrated into care with minimal disruption. 22 This pilot study provides preliminary evidence on the feasibility of recruiting patients in advance of visits and the effectiveness of available decisions aids, and highlights a tradeoff between length, usage, and outcomes.
Footnotes
Acknowledgements
We acknowledge the valuable contributions of our research coordinator, Emily Wendell, and the members of the Health Decision Sciences Center’s Patient Advisory Committee.
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Sepucha (PI) has received salary support as a medical editor for the Informed Medical Decisions Foundation (IMDF). From 1997 to 2014, the IMDF was associated with Health Dialog; from 2014 to 2017, the IMDF was part of Healthwise; and in 2017, the IMDF became part of Massachusetts General Hospital. Dr. Freiberg reports other support from Zimmer Biomet, ArthroSurface, CeramTec, and Orthopaedic Technology Group, outside the submitted work. Dr. Bedair reports personal fees from Smith & Nephew, personal fees from Conformis, outside the submitted work. Dr. Dwyer and Ms. Mangla declare that they have no competing interests.
Financial support for this study was provided in part by a grant from The Gordon and Betty Moore Foundation (#3940) and in part through the Patient-Centered Outcomes Research Institute (PCORI) award (CDR#1503-28799). The statements presented in this article are solely the responsibility of the authors and do not necessarily represent the views of the PCORI, its Board of Governors, or Methodology Committee. The funding agreements ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.
Part of this work was presented at the 2016 Society for Medical Decision Making’s Annual North American Conference in Vancouver, Canada.
Authors’ Note
Investigators interested in accessing the surveys, codebooks, and/or data should contact the corresponding author. Access to the data is limited and subject to the policies of Partners Human Research Committee and the approved institutional review board protocol.
