Abstract
Lung cancer screening with annual low-dose computed tomography (LDCT) scans is now recommended for persons 55 to 80 y of age who have a 30 or more pack-year smoking history and currently smoke or have quit within the past 15 y. 1 Annual screening is associated with a survival benefit but is also associated with potential harms, most notably overdiagnosis 2 and a high false-positive rate.1–3 Therefore, effective communication of the probabilities associated with both the expected benefits and the harms of screening is essential to ensure that patients understand possible downstream effects, such as additional imaging tests and invasive procedures for benign lesions. In the Center for Medicare and Medicaid Services (CMS) memo published in February 2015 addressing lung cancer screening for appropriate beneficiaries with LDCT, 4 a shared decision-making visit is cited as a required criterion for coverage and must include at least 1 decision aid; however, CMS has not recommended a specific tool.
A large body of research has documented the difficulties associated with effectively communicating probabilistic information.5–7 In medicine, one of the most significant barriers to informed decision making is the tendency to overweight small probabilities. As a result, many studies have sought to develop improved methods to improve patients’ understanding of risk. 8 Notably, Fagerlin and others 9 recently outlined a series of recommendations to improve risk communication, which included presenting absolute risks instead of relative risks and using icon arrays.
To date, all approaches used to improve patients’ understanding of probabilities use descriptive formats; however, a growing literature suggests that risk might also be effectively communicated though experience. Shaffer and others 10 found, for example, that narratives focused on describing other patients’ experiences resulted in improved decision-related measures, including confidence in the ability to make an informed choice and satisfaction with the decision process. However, whether testimonials result in an increased likelihood of patients making informed choices that are concordant with their values is not yet known. 11 More recently, investigators have investigated the effects of experiencing probability distributions on decision making. Hertwig and others 12 demonstrated that decisions made based on descriptions versus those made based on personal experiences lead to significantly different choices. Moreover, whereas description leads to overestimation of rare risks, experience generally results in the opposite effect, as rare risks are seldom encountered.
The majority of the experiments documenting this effect have asked students to engage in gambling tasks in which participants infer the expected probabilities of competing outcomes by performing repeated sampling tasks. 13 Conversely, Tyszka and Sawicki 14 compared the impact of descriptive and experienced-based probability formats in a health care scenario. They asked female students to consider prenatal genetic testing, in which they experienced risk by viewing a series of photographs of children with and without Down syndrome. The authors found that worry about the genetic disease was lower among those exposed to the experienced versus description format. This effect may have been due to superior learning of risks or lack of exposure to infrequent risks.
Given this background, we sought to examine the effect of descriptive versus experienced risk formats among patients. Specifically, we compared the effects of 3 modes of risk presentation on patients’ objective knowledge, beliefs, endorsement of screening for heavy smokers, and personal preference to undergo lung cancer screening. Based on previous research, we expected that participants randomized to the experienced format would have greater knowledge scores, less favorable beliefs, and be less likely to endorse screening or to choose screening for themselves compared with those randomized to the descriptive formats.
Methods
Participants and Procedures
Participants were recruited from an outpatient pulmonary practice during their reminder telephone call. All English-speaking participants, who were not being scheduled for follow-up of a pulmonary nodule and who did not have a history of lung cancer, were asked if they were willing to come to their follow-up appointment 30 min early to participate in the study. Upon arrival, the study was described in detail, and verbal consent was obtained.
Participants were subsequently randomized to learn about lung cancer screening using 1 of 3 probability formats: numbers only, numbers and corresponding icon arrays, numbers and a set of slides illustrating LDCT scans (presented in the supplemental online appendix). The LDCT scans displayed the number of normal scans, false-positive scans with benign lung nodules found (referred to as “false alarms”), cancers found leading to a life saved, and cancers found leading to death despite early identification and treatment, for 250 people in random order. Each slide was shown for 1 s, for a total viewing time of 4 min. The transition time between slides was chosen based on feedback obtained from participants during pilot testing. Although the National Cancer Institute’s “Patient and Physician Guide” related to lung cancer screening presents outcomes for a population of 1000 persons, 15 we chose to use 250 as the denominator for this study to decrease the time required for the experience task. We could not use 100 as the denominator because of the rarity of expected events and the need to present whole numbers. During the viewing period, participants were seated in a quiet private room in front of a computer screen and directly observed by a medical assistant.
Measures
We measured objective knowledge using 3 questions: 1) If all 250 people get CT scans to screen for lung cancer: A) No one will die of lung cancer, B) A few people will still die of lung cancer but most will be saved, C) Equal numbers of people will die of lung cancer as those who will be saved, D) Most people will still die of lung cancer, but someone will be saved (correct response = D). 2) Compared with the number of people with lung cancers found by screening, how many people will have a false alarm? A) Many more, B) A few more, C) A few less, D) A lot less (correct response = A). 3) Of all the people screened, about how many will have a false alarm? A) 5 in 100 (5%), B) 20 in 100 (20%), C) 35 in 100 (35%), D) 50 in 100 (50%) (correct response = C). Forty-two participants (16.5%) answered the first knowledge question correctly, 143 (56.3%) answered the second question correctly, and 98 (38.6%) answered the third question correctly. Given the ambiguity of the first knowledge question, only the latter two were used in the analyses. Scores were summed and treated as a continuous. Beliefs related to lung cancer screening were measured by summing the following four items measured on 5-point scales ranging from 1 = strongly agree to 5 = strongly disagree: “Screening is always the right thing to do because it catches cancer early” (reverse scored); “Screening may not be the right thing to do if there are too many false alarms”; “Screening is worth it even if it saves 1 in a 1,000,000 lives” (reverse scored); “Screening may not be the right thing to do if it leads to over diagnosis of spots on the lung that would not have caused any harm during a person’s lifetime.” Higher scores indicate more favorable beliefs toward screening.
We measured 2 types of preferences: 1) Endorsement of screening was measured by asking participants: Based on the information given to you, do you think heavy smokers should have screening LDCT scans? (Yes, No, Not Sure). No (n = 12) and not sure (n = 23) were collapsed into a single group because the numbers of participants per category was too small to analyze separately. 2) Personal choice predisposition was measured using the following question rated on an 11-point numeric rating scale “What do you think right now about having a CT scan to screen for lung cancer each year?” anchored by “I am certain that I don’t want to be screened” and “I am certain that I do want to be screened,” with “Unsure” in the middle. 16 Among participants who did not previously smoke, the question was preceded by, “Imagine that you were a heavy smoker.”
Perceived chance of developing lung cancer and worry related to developing lung cancer were also measured on 11-point numeric rating scales. Overall health status was measured using a 5-point scale ranging from excellent to poor. Objective numeracy was measured by using 4 items from a validated numeracy scale. 17
Analyses
Analyses were conducted using SAS version 9.3. We used an analysis of variance to detect differences in continuous variable (knowledge, beliefs, choice predisposition) across the 3 probability formats with Tukey’s procedure to examine specific pairwise comparisons. The chi-square test was used to detect differences in endorsement across the 3 formats. t Tests and Pearson’s correlation coefficients were used to detect associations between choice predisposition with categorical and continuous variables, respectively. We also performed exploratory analyses to examine whether probability format modified the relationship between knowledge, beliefs, worry about developing lung cancer, and perceived chances of developing lung cancer with choice predisposition. Interactions in which P < 0.1 are reported. Choice predisposition was highly skewed and therefore log transformed.
We calculated that 246 participants were needed to detect an effect size of 0.20 or greater assuming α = 0.05, β = 0.80. A greater number of participants were enrolled (N = 254) because we did not cancel interviews once patients had agreed to participate over the phone prior to their interview.
Results
Participants’ Characteristics
Of the 431 invited to participate, 276 participants agreed to be interviewed. Two hundred fifty-four arrived for their appointment, and all consented to participate. Eighty-four were randomized to view the numbers-only format, 86 to view the numbers + icon array format, and 83 to view the numbers + slide presentation. The mean (s) age of the study sample was 60.9 (8.8) years; 137 (54.2%) were woman, 88 (34.7%) were currently employed, and 143 (56.3%) were current or past smokers of whom 73 (28.7%) had a 30 or greater pack-year smoking history. Mean (SD) numeracy was 1.8 (1.0; maximum possible score = 4). The demographic and clinical characteristics by probability format are described in Table 1. We found no statistically significant differences in patient characteristics across the 3 groups; however, the proportion of never smokers was higher among participants randomized to the numbers + icon array format.
Subjects’ Characteristics by Probability Representation Format
P values for F and chi-square tests for continuous and categorical variables, respectively.
Differences in knowledge, beliefs, endorsement of screening for heavy smokers, and personal choice predisposition by probability format are illustrated in Table 2. Average knowledge differed between the 3 formats (overall difference between means, P = 0.001). Knowledge was greater in the numbers + icon array and the numbers + experience formats when compared with the numbers-only format (difference between means [95% confidence interval {CI}]= 0.5 [0.2–0.7] and 0.3 [0.01–0.6], respectively).
Differences in Knowledge, Beliefs, Endorsement, and Choice Predisposition by Probability Representation Format a
Columns that do not share a superscripted letter are statistically different.
Because choice predisposition was highly skewed, we also performed nonparametric statistics (Kruskal-Wallis test P = 0.4).
The percentage of participants responding correctly to the second knowledge question was higher for those randomized to the numbers + icon array and numbers + experienced format compared with the numbers-only format (Table 3). In contrast, the percentage of participants responding correctly to the third knowledge question was higher only for those randomized to the numbers + icon array format compared with the numbers-only format. Objective numeracy did not modify the relationship between probability format and knowledge overall or knowledge for any specific item.
Percentage Correct Answers for Each Knowledge Question by Probability Representation Format a
Columns that do not share a superscripted letter are statistically different.
Beliefs related to screening also differed by format (overall difference between means, P = 0.01), with the least favorable attitude toward screening observed in participants randomized to the numbers + icon array format (Table 2). Beliefs were more favorable among participants randomized to the numbers + experienced format compared with the numbers + icon array format (difference between means [95% CI] = 1.6 [0.4–2.8]). Objective numeracy did not modify the relationship between probability format and beliefs.
Smoking (whether measured as current, past, or never smokers using dummy variables or by whether they met criteria for screening by pack years) was unrelated to either knowledge or beliefs and did not alter the associations between probability format and either dependent variable.
Differences in participants’ endorsement of screening (P = 0.4) and choice predisposition (P = 0.6) across probability format mirrored those of beliefs but were not statistically significant (Table 2).
On average across probability formats, greater choice predisposition was correlated with more favorable beliefs (r = 0.4, P < 0.0001), increased worry about developing lung cancer (r = 0.2, P = 0.003), and greater perceived chances of developing lung cancer (r = 0.2, P = 0.0002); it was not associated with knowledge (r = −0.1, P = 0.3). The interaction between beliefs and probability format was borderline significant (P = 0.06) and that between worry and probability format was significant (P = 0.04). The correlation between beliefs and worry with choice predisposition within each probability format is described in Table 4. More favorable beliefs were associated with stronger choice predisposition in each format, but the association was somewhat weaker in participants randomized to the experienced format compared with the other 2 groups. Increased worry about developing lung cancer, on the other hand, was associated with greater choice predisposition among participants randomized to the numbers + icon array and numbers + experienced formats but not among those randomized to the numbers-only format. Probability format did not modify the positive relationship between perceived chance of developing cancer and choice predisposition.
Correlations between Beliefs and Worry Related to Developing Lung Cancer with Choice Predisposition by Probability Representation Format
Discussion
In this study, we compared the effect of probability format on patients’ knowledge, beliefs, and preferences related to lung cancer screening and found that the numbers + icon array and numbers + experienced formats were associated with higher knowledge scores compared with the numbers-only format. The improvement in knowledge among participants presented with the addition of an icon array is consistent with prior studies demonstrating the benefits of using this format. 9 Improved knowledge, however, was not associated with choice predisposition. Unlike some previous studies, we found no interaction between objective numeracy and format, suggesting that participants’ knowledge improved to a similar degree regardless of numeracy levels. We did not measure graphic literacy, which theoretically may also interact with probability format. 18
Contrary to what we expected, the experienced format increased propensity toward screening compared with the numbers + icon array format as indicated by more favorable beliefs, as well as nonsignificant trends toward stronger choice predisposition and endorsement. Previous studies have found that participants have lower estimates of rare events when presented with experienced risk compared with descriptive risk formats. Thus, we were surprised to find that propensity toward screening was more favorable among those randomized to the experienced condition compared with the numbers + icon array format, since we expected that these participants would better appreciate the very small number of patients who actually benefit from screening compared with the number who have normal or false-positive LDCT scans. There are several possible reasons for our findings. First, the use of an advancing series of images may have actually lowered patients’ ability to appreciate the proportion of patients whose lives are saved by screening. This explanation is consistent with previous research demonstrating that interactive risk presentation formats can impede understanding of probabilistic information. 19 Second, we assumed that normal LDCT scans would be associated with disutility (i.e., a wasted resource); however, patients may be reassured by having a normal test and therefore may have viewed each normal LDCT slide as a positive outcome. If true, given that images are known to have a greater impact on decision making compared with descriptive text, the experienced task might have increased propensity for screening by reinforcing the positive reaction to a normal test result at least among some individuals. This possibility is supported by a recent qualitative study describing smokers’ misperceptions related to lung cancer screening including the belief that normal test results indicate that they are among the fortunate minority who will not experience harm from smoking. 20 It would be interesting to examine how experienced formats affect patients’ perceptions related to the adverse events and benefits associated with treatment, because unlike in screening, the positive or negative utility of each outcome is clear.
We also assumed that the experienced format would enable patients to better appreciate the large number of false-positives associated with LDCT. While the experienced task did increase the number of correct responses to the false alarm knowledge question (as compared with the numbers-only format), this knowledge did not affect propensity toward screening. This finding may reflect a patient perspective that false-positives are not important; this interpretation is consistent with the lack of anxiety, distress, worry, or health-related quality of life found in patients receiving false-positive results after having undergone screening.21,22
In exploratory analysis, we found that worry about developing lung cancer was correlated with choice predisposition among participants randomized to the numbers + icon array and numbers + experienced formats but not among those in the numbers-only format. These results suggest that, although viewing the icon array or set of slides did not increase the mean level of worry among subjects relative to the presumably more abstract numbers-only format, 23 viewing either format may have motivated the use of affect in subsequent evaluations. 24
We chose to present participants with pictures based on our need to effectively communicate distinct outcomes while minimizing cognitive burden. Tyszka and Sawicki 14 used a similar approach but showed participants familiar images (babies’ faces). In this study, we showed pictures of LDCT scans since these are the direct results of the screening process. All participants were familiarized with the images and their meaning prior to engaging in the experienced task. However, the present images were certainly less familiar than baby faces and may not have conveyed the meaning we intended. Although images can arouse greater emotional meaning than text (or numbers)14,25 and this greater arousal can promote greater learning, 26 our unfamiliar scans may not have done so. Alternatively, the complex images may simply have increased the cognitive burden on participants and reduced their overall comprehension of the situation, only part of which was captured by our knowledge items. 27
It is difficult to compare our experiment with previous experienced risk studies since the latter have included 2 possible outcomes, whereas in this study, we asked patients to consider 4 different LDCT scans (depicted in the supplemental online appendix). It is possible that a simpler experienced condition may have led to different results. In addition, most previous experiments have required participants to actively sample from a baseline distribution as many times as they wished, whereas our task, like the one used by Tyszka and Sawicki, 14 forced participants to view a single random distribution of all possible events, thus ensuring that each participant viewed the expected number of each outcome.
There are several limitations of this study, which can be addressed in future research projects. We chose to run the study prior to patients seeing their physicians so as to avoid the influence of the visit on participants’ responses; thus, the length of the survey had to be limited in order not to interrupt clinic flow. Because there are no known validated knowledge or endorsement measures, both were developed for this study. Moreover, a greater number of knowledge questions may have increased the distribution of this variable and resulted in greater power to detect associations between knowledge and preferences. Our experience task, while mirroring the distribution of actual outcomes associated with lung cancer screening, may have been overly complex, and our results may not generalize to simpler tasks asking participants to consider fewer discrete events. Simpler experiments, however, may be of limited use in screening decisions in which multiple outcomes usually need to be considered. The value of the present approach may also be constrained by the respondents’ ability to attend to an experienced condition with a large denominator. For this reason, we presented patients with a distribution of LDCT outcomes in a population of 250 (as opposed to a population of 1000 used in the National Cancer Institute’s “Patient and Physician Guide” from the National Lung Cancer Screening Trial 15 )1,3; however, our task still required participants to attend to 250 PowerPoint slides over 4 min. Participants in the numbers-only and numbers + icon array groups did not view LDCT scan images, and thus, we did not control for the influence of seeing the images on participants’ responses or of the longer time required to view the slides compared with seeing the information in the other 2 formats. In addition, because our primary hypothesis was to compare experienced versus descriptive formats, we included participants who were not eligible for lung cancer screening; however, the patients’ approaches to even a hypothetical scenario may differ depending on their eligibility for screening and the perceived relevance of the task.
In this study, an experienced risk format was not superior to an icon array in improving participants’ knowledge about a subset of outcomes related to lung cancer screening. Moreover, we found that contrary to expectations, the numbers + experienced format may have increased propensity toward screening compared with the numbers + icon array format and particularly with respect to positive beliefs about screening. Given what we have learned from this study, future research examining the impact of experienced formats should 1) focus on health-related decisions involving more frequent outcomes to avoid large denominators requiring lengthy experience tasks, 2) consider choices for which patients do not hold strong beliefs (as they do for decisions such as cancer screening and vaccines), and 3) examine choices in which the direction of utility related to each outcome is clear-cut.
Footnotes
Acknowledgements
The authors thank Amanda Kahn, Darly George, and Adam Oelberg for their time and effort screening potential subjects, performing data acquisition, and data entry.
Financial support for this study was provided in part by National Science Foundation NSF SES-1047757 to the second author and the National Institute of Arthritis and Musculoskeletal and Skin Diseases, part of the National Institutes of Health, under award number AR060231-01 (to LF). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
