Validation of SURE,a Four-Item Clinical Checklist for Detecting Decisional Conflict in Patients

Abstract

Background: We sought to determine the psychometric properties of SURE, a 4-item checklist designed to screen for clinically significant decisional conflict in clinical practice. Methods: This study was a secondary analysis of a clustered randomized trial assessing the effect of DECISION+2, a 2-hour online tutorial followed by a 2-hour interactive workshop on shared decision making, on decisions to use antibiotics for acute respiratory infections. Patients completed SURE and also the Decisional Conflict Scale (DCS), as the gold standard, after consultation. We evaluated internal consistency of SURE using the Kuder-Richardson 20 coefficient (KR-20). We compared DCS and SURE scores using the Spearman correlation coefficient. We assessed sensitivity and specificity of SURE scores (cut-off score ≤3 out of 4) by identifying patients with and without clinically significant decisional conflict (DCS score >37.5 on a scale of 0–100). Results: Of the 712 patients recruited during the trial, 654 completed both tools. SURE scores showed adequate internal consistency (KR-20 coefficient of 0.7). There was a significant correlation between DCS and SURE scores (Spearman’s ρ = −0.45, P < 0.0001). The prevalence of clinically significant decisional conflict as estimated by the DCS was 5.2% (95% CI 3.7–7.3). Sensitivity and specificity of SURE ≤3 were 94.1% (95% CI 78.9–99.0) and 89.8% (95% CI 87.1–92.0), respectively. Conclusions: SURE shows adequate psychometric properties in a primary care population with a low prevalence of clinically significant decisional conflict. SURE has the potential to be a useful screening tool for practitioners, responding to the growing need for detecting clinically significant decisional conflict in patients.

Keywords

shared decision making decisional conflict clinical checklist SURE sensitivity and specificity

When patients have to make decisions without adequate knowledge, unrelated to their values and preferences, and under undue pressure from others, the result can be decisional conflict or personal uncertainty about which option to choose.^1–3 Unresolved decisional conflict can lead to adverse effects in patients, such as physical tension (i.e., increased heart rate, muscle tension, restlessness, etc.) and emotional distress.² Moreover, downstream effects of unresolved decisional conflict include decisional regret and, in the case of deleterious outcomes, blaming the health care provider.^4,5 Screening for decisional conflict in patients is thus an important shared decision-making competency⁶ and a critical step in ensuring that informed consent is truly informed and good decisions are made.⁷

The Decisional Conflict Scale (DCS) is the tool most commonly used to assess the impact of patient decision aids.⁸ The DCS assesses decisional conflict in patients and has been used in a variety of clinical settings.^8–13 It comprises 5 subscales (informed, values clarity, support, uncertainty, and effective decision subscales),^3,14 and we know it to be valid and reliable.¹⁵ The DCS total score is calculated out of 100, where a score lower than 25 is associated with effectively doing what was decided and, according to the user’s manual, a score over 37.5 is associated with decision delay and feeling unsure about the implementation of the decision.¹⁴ However, the DCS is not optimal for routine use by health care providers in clinical practice because it takes too long to administer.¹⁶

As an alternative, SURE, a simple checklist based on the DCS, consists of only 4 items with a dichotomous response scale (yes or no). SURE is designed specifically to screen for decisional conflict associated with a DCS score greater than 37.5 out of 100.¹⁷ SURE presents 4 of the 5 DCS constructs, with 1 item for each (see Table 1). The DCS “effective decision” subscale was left out, as SURE is intended to appraise the readiness of a patient to make a decision as well as his or her comfort with making the decision. To maximize SURE’s sensitivity and negative predictive value, its authors designed it to consider a patient as positively identified if he or she answered “no” to 1 of the 4 questions.^17–19 It is thus intended to be used by practitioners during the deliberation period. The format of SURE, with its 4 simple questions answered by yes or no, was inspired by the CAGE test, a 4-item checklist widely used in primary care to identify patients with potential alcohol addiction problems.^17,20–22 In an earlier study, SURE was tested with a cohort of 123 pregnant women between 18 and 34 years old considering prenatal screening for Down syndrome.¹⁷ Results showed that SURE had acceptable psychometric properties and was suitable for detecting clinically significant decisional conflict in patients facing clinical decisions in primary care.¹⁷ However, no study has yet assessed SURE in a broader and heterogeneous group of patients.

Table 1

The SURE Tool

SURE Acronym	Items	Answer
1. Sure of myself	Do you feel sure about the best choice for you?	Yes/No
2. Understand information	Do you know the benefits and risks of each option?	Yes/No
3. Risks/benefits ratio	Are you clear about which benefits and risks matter most to you?	Yes/No
4. Encouragement	Do you have enough support and advice to make a choice?	Yes/No

©Légaré and O’Connor 2008.

The purpose of this study was to assess the psychometric properties of SURE (i.e., its internal consistency and diagnostic validity) in a large and heterogeneous group of patients consulting for possible antibiotic treatment of acute respiratory infection (ARI) in primary care settings.

Methods

Study Context

This study is a secondary analysis of a clustered randomized trial aimed at assessing the effect of DECISION+2, a 2-hour online tutorial followed by a 2-hour interactive workshop about shared decision making (SDM), on the proportion of ARI patients reporting a decision to use antibiotics immediately after consultation.²³ DECISION+2 was designed to help family physicians (FPs) engage patients in decision making about the use of antibiotics to treat ARIs and thus to address overuse.²⁴ One of the steps in the shared decision-making process is detecting whether patients are comfortable with their decisions, and SURE is thought to be a potentially useful tool for this step.

Participants and Recruitment

Details of the main trial DECISION+2 protocol and results have been published elsewhere.^23,25 Briefly, patients were recruited in 9 family practice teaching units (FPTUs); 5 of the FPTUs were randomly allocated to the experimental group (i.e., they received the DECISION+2 training) and 4 to the control group (i.e., usual care). A research agent posted in each of the 9 participating FPTUs recruited patients who met 2 inclusion criteria, namely 1) patients with a positive diagnosis of ARI and 2) patients for whom the use of antibiotics was subsequently considered either by the patient or by the physician during the visit. Patients also had to be 18 years old or older or accompanied by a parent or legal guardian. They were excluded if they presented a condition requiring an emergency response. The study received the approval of the local research ethics board, and all patients signed an informed consent form.²³

Data Collection

As secondary outcomes of the DECISION+2 trial, we assessed patients’ decisional conflict with the DCS and SURE in a self-administered questionnaire that was given to patients after the clinical encounter with their FP. Patients first completed the 4-item SURE test (Table 1) followed by the DCS test, which includes 16 items each measured on a 5-point Likert scale (0 = strongly agree to 4 = strongly disagree). Patients were also asked what decision had been made about antibiotics to treat their ARI (no prescription, immediate prescription, or prescription delayed). We collected sociodemographic data in a questionnaire that patients completed before the medical encounter. Two weeks after the encounter, participants were called by a research agent to assess their decisional regret using the Decisional Regret Scale,⁴ their adherence to the decision, and whether they had consulted again for the same reason within those 2 weeks.

Data Analysis

We assessed the psychometric properties of SURE by looking at the internal consistency, correlation and association with the DCS, and diagnostic validity of the scores. Internal consistency was established using the Kuder-Richardson 20 (KR-20) coefficient for dichotomous items. The correlation between the DCS continuous scores (0–100) and SURE ordinal scores (0–4) was calculated using the Spearman rank-order correlation. We used the Kruskal-Wallis rank test to determine whether DCS scores were related to SURE scores.²⁶

The sensitivity and specificity of SURE were estimated by comparing classification by SURE with classification by DCS. SURE scores could range from 0 to 4. A “yes” answer to each item was given a 1 value and a “no” was given 0. A score of 3 or less out of 4 was considered a positive test result (i.e., an indicator of clinically significant decisional conflict).¹⁷ A perfect score of 4 on the SURE test means that the individual should not be experiencing clinically significant decisional conflict. The DCS responses were scored so that total scores would range from 0 to 100, where higher scores were associated with more decisional conflict. Therefore, a negative correlation is to be expected between SURE and the DCS. Although the dichotomization of a continuous variable such as the DCS is usually not encouraged,^27–29 many studies agree that the 37.5 DCS cut-off point (out of 100) is suitable for distinguishing between patients with and without clinically significant decisional conflict.^{14,17,30–33} We thus used this threshold of the DCS as the gold standard to compute accuracy, sensitivity, specificity, predictive values, and positive and negative likelihood ratios of the SURE test. The DCS also allowed us to estimate the prevalence of clinically significant decisional conflict in our study population, as we had no previous data on this issue.

On an exploratory basis, we also performed Fisher’s exact test to compare the sensitivity results of SURE on various patient subgroups that had been diagnosed with clinically significant decisional conflict using the DCS. As women generally experience more decisional conflict,³⁴ we compared female with male patients. To explore whether the sensitivity of SURE was affected by the physician’s approach to shared decision making, we also conducted the same analysis according to whether the FP consulted had been exposed to the DECISION+2 training program. We also explored whether the clinical decision made by the patient (i.e., whether the patient reported an immediate prescription for antibiotics) was associated with sensitivity of SURE. We looked at possible associations between clinically significant decisional conflict as screened by SURE and patient outcomes measured at 2 weeks, as the literature shows that decisional conflict is associated with decisional regret and discontinuance of treatment.³⁵ We applied Student’s t test for decisional regret and Fisher’s exact test for adherence to the decision made. We performed a χ² test to assess a possible correlation between clinically significant decisional conflict screened by SURE and consultation for the same reason within the 2 weeks, as patients who were uncertain about their antibiotic decision a priori would probably consult again in the next few days. All calculations were performed using SAS version 9.2.

Results

Participant Characteristics

Recruitment began in July 2010 and ended in April 2011. We recruited 712 eligible patients. Table 2 presents participants’ sociodemographic characteristics. Of the 712 patients recruited for the study, 672 (94.4%) completed all SURE items, 689 patients (96.8%) filled out the entire DCS, and 654 (91.8%) completed them both. Figure 1 presents the STARD (STAndards for the Reporting of Diagnostic accuracy studies) statement flow diagram of participating patients and the dichotomized data obtained for both tests (SURE and DCS).

Table 2

Sociodemographic Characteristics of Participants

Characteristic
Women, n/N (%)	445/691 (64)
Adults, n/N (%)^a	495/696 (71)
Adults’ age, years, ± s	42.0 ± 15.0
Children, n/N (%)	201/696 (29)
Children’s age, years, ± s	5.0 ± 3.8
Household size, ± s	3.4 ± 1.2
Participants presenting at least 1 chronic disease, n/N (%)	101/712 (14)
Participants with family income ≥Can$ 45,000/year, n/N (%)	283/481 (59)
Participants currently working, n/N (%)	365/508 (72)
Participants with a college or university degree, n/N (%)	308/511 (60)

Adults are 18 years or older.

Figure 1

STARD Statement flow diagram of participating patients. DC, decisional conflict; DCS, Decisional Conflict Scale.

Decisional Conflict Scores

Of the 654 patients who completed both tools, 34 presented a clinically significant decisional conflict as revealed by the DCS, which estimated a prevalence of 5.2% (95% CI 3.7–7.3). Of all patients who answered the entire SURE test (n = 672), 574 (80.9%) scored 4 out of 4; 49 (7.3%) scored 3; 26 (4.2%) scored 2; 19 (7%) scored 1; and 4 (0.6%) scored 0. SURE items 2 “Do you know the benefits and risks of each option?” and 3 “Are you clear about which benefits and risks matter most to you?” each accounted for 35.1% (61/174 each) of all items that were rated “no” by patients.

Psychometric Properties

The internal consistency of the SURE scores was adequate (KR-20 coefficient = 0.70). The Spearman correlation test showed a moderately strong negative association between DCS and SURE scores (ρ = −0.45, P < 0.0001). Figure 2 illustrates the results from the Kruskal-Wallis rank test, showing that DCS scores were significantly associated with SURE scores (P < 0.0001).

Figure 2

DCS score descriptive statistics relative to SURE scores. DCS scores range from 0 to 100, with a higher score being associated with more decisional conflict. The bottom and top of boxes are the 25th and 75th percentiles (the lower and upper quartiles, respectively), and the band near the middle of the box is the median. The minimum and maximum scores are represented by the ends of the whiskers. Means are represented by diamonds.

The overall accuracy of SURE was 90.1% (95% CI 87.8–92.4), so that 9 patients out of 10 were correctly screened as presenting clinically significant decisional conflict or not. Sensitivity was 94.3% (95% CI 78.9–99.0), meaning that out of 100 patients presenting with clinically significant decisional conflict, the SURE test would correctly identify 94 of them. Specificity was 89.8% (95% CI 87.1–92.0), so that out of 100 patients presenting no clinically significant decisional conflict, SURE would falsely identify it in 10 of them. Given our sample’s decisional conflict prevalence, the negative predictive value was 99.6% (95% CI 98.6–99.9)—a patient answering “yes” to all 4 SURE questions was almost never experiencing clinically significant decisional conflict. In contrast, the positive predictive value was 33.7% (95% CI 24.5–44.2), meaning that only 1 person out of 3 who tested positive with SURE would in fact present clinically significant decisional conflict. The positive likelihood ratio was moderately high (9.26, 95% CI 7.22–11.88), meaning that with a positive SURE test, a patient had 9 times greater odds of experiencing clinically significant decisional conflict. The negative likelihood ratio was clinically more significant (0.07, 95% CI 0.02–0.25), meaning that for a negative SURE test, a patient is 14 times more likely not to experience clinically significant decisional conflict than to experience it.

Exploratory Analysis

The sensitivity of SURE did not differ with patient gender (P = 1) or with FPs’ exposure to DECISION+2 (P = 1) and was not statistically significant among patients who reported a decision of immediate prescription of antibiotics compared with those who reported decision of delayed or no prescription at all (P = 0.09) (see Table 3). Patients screened with clinically significant decisional conflict experienced more decisional regret about their choice to take antibiotics or not 2 weeks after the consultation (P < 0.01). Adherence to the decision and consultation for the same reason within the 2 weeks did not correlate with clinically significant decisional conflict as assessed by SURE (P = 1.0 and 0.97, respectively).

Table 3

Sensitivity (%) of SURE Assessed for Subgroups

Subgroup	Sensitivity (95% CI)
Female (n = 24)	91.7 (71.5–98.5)
Male (n = 9)^a	100 (62.9–100)
Immediate antibiotics prescription reported (n = 11)	81.8 (47.8–96.8)
No immediate antibiotics prescription reported (n = 23)	100 (82.2–100)
FP consulted had not been exposed to DECISION+2 (n = 3)	100 (31.0–100)
FP consulted had been exposed to DECISION+2 (n = 31)	93.5 (77.2–98.9)

Note: CI =confidence interval; FP = family physician.

Data missing for 1 subject.

Discussion

SURE shows adequate psychometric properties in a primary care population with a low prevalence of clinically significant decisional conflict, such as in the context of antibiotic use to treat ARIs. SURE’s internal consistency is acceptable, and the association between DCS and SURE scores is substantial. If we consider DCS >37.5 as the standard for clinically significant decisional conflict, classification based on SURE <4 shows satisfactory accuracy, sensitivity, and specificity, providing some diagnostic validity evidence. Our study is also the first to estimate the prevalence of clinically significant decisional conflict in a specific primary care situation, namely the decision to use antibiotics to treat ARIs. Considering these findings, we make 3 important observations.

First, although this was not the main objective of our study, using the DCS cut-off of >37.5 out of 100 gave a low estimate of the prevalence of clinically significant decisional conflict among patients having to choose whether to use antibiotics to treat ARI. When we compared our estimated prevalence of clinically significant decisional conflict to results in the literature, we realized that even judging by studies that all used the same >37.5/100 cut-off, prevalence is highly variable and depends on the nature of the medical decision and factors associated with the process of making that decision.^17,30–33 For example, one study on prenatal screening for Down syndrome in primary care established a 6% prevalence of clinically significant decisional conflict,¹⁷ while another study of men considering vasectomy as a permanent birth control method but still undecided found a 52% prevalence of clinically significant decisional conflict.³¹

However, there is no doubt that when it comes to deciding whether to use antibiotics to treat an ARI, an estimated 5% prevalence of clinically significant decisional conflict among the population facing that decision over the course of a year can have major consequences. According to the National Ambulatory Medical Care Survey in the United States,³⁶ 30,916,000 office visits in 2006 were for ARIs (excluding pharyngitis). Extrapolating the 5% prevalence found in our study, we could surmise that 1,545,000 patients experienced clinically significant decisional conflict about whether to take antibiotics to treat their ARI over a 1-year period. From a public health perspective, the absolute number of patients experiencing clinically significant decisional conflict should be a matter of concern for health care providers and should prompt them to identify and provide further support to such patients.

Second, to be appropriate for clinical use and useful to busy primary care physicians, clinical checklists must be brief and case-finding²⁰; otherwise, they will not be successfully implemented.³⁷ Like the CAGE test, widely used to screen for alcohol abuse or dependence,²² SURE is user-friendly and its psychometric properties make it relevant for identifying patients with high risks of downstream effects from their clinically significant decisional conflict. Moreover, our exploratory analysis found no statistically significant differences between the patient subgroups we assessed, which is encouraging.

Third, being able to detect clinically significant decisional conflict in patients has become important for health care providers in their concern to ensure decision quality and informed consent.⁷ In this regard, since 2009 the Registered Nurses’ Association of Ontario, Canada, has recommended use of SURE to detect potential clinically significant decisional conflict as part of the decision support to be given to patients with chronic kidney disease.^38,39 In some medical and nursing schools in North America, students are trained to identify clinically significant decisional conflict in patients during simulated patient experiences.^40–42

Consistent with literature establishing that patients experiencing decisional conflict are more likely to regret their decision,^4,9,35,43,44 our study found that patients with clinically significant decisional conflict as detected by SURE experienced more decisional regret about their choice of whether to take antibiotics 2 weeks after the consultation than those without detected decisional conflict. However, we found no association between adherence to the decision and consultation for the same reason within the 2 following weeks. These analyses were exploratory and were based on earlier studies that showed an association between decisional conflict and discontinuance of treatment.^35,45 Our results may be partly explained by the small sample size, but they may also be explained by the nature of the decision being studied. Since taking antibiotics is not very constraining and the symptoms are usually of short duration, patients with clinically significant decisional conflict may not have felt the need to change the management of their ARI as established during the consultation. Most of the earlier studies that found an association between decisional conflict and discontinuance of the treatment were related to hormone replacement therapy for menopause symptoms, a long-term situation for which reconsideration of the decision may be more likely to occur.^35,45

Limitations

This study has some limitations. Our results may be generalizable only to populations with a low prevalence of clinically significant decisional conflict and perhaps only to those in a similar clinical context. As this is only the second study reporting data on SURE with a large sample size, further studies should assess SURE’s psychometric properties, particularly among populations and in clinical situations where clinically significant decisional conflict is highly prevalent, to confirm that our results are more widely applicable. We also had a limited sample size for our exploratory analysis, as we were analyzing only patients expressing clinically significant decisional conflict according to DCS (n = 34) or SURE (n = 89). These results must therefore be interpreted with caution.

Another important limitation is that patients were not blinded to the 2 tests. In the same questionnaire they answered questions from the SURE tool and the DCS. Although this situation was not ideal, it was unavoidable as decisional conflict is a state (i.e., a person’s characteristic at a particular moment; a state is not static, it is bound to change over time) rather than a trait (i.e., a personal characteristic that is permanent or long lasting). As such, patients had to complete the 2 tests simultaneously if we wanted to get the real picture of their state. We tried to minimize this limitation by placing the SURE tool before the more numerous DCS questions so that patients were more likely to complete SURE and to avoid the obvious item repetition, but this could still have resulted in an overestimation of SURE’s psychometric results. A more effective way could have been to distribute SURE and DCS questions randomly throughout the questionnaires.

Also, we used all the available literature to determine the best threshold for clinically significant decisional conflict, in order to assess SURE’s sensitivity and specificity. However, we are aware that a certain lack of evidence still exists concerning the interpretation of DCS scores associated with downstream effects.⁴⁶ Although treating DCS >37.5 as the standard for clinically significant decisional conflict showed satisfactory sensitivity and specificity, further studies are needed for a full comprehension of DCS scores and their relations to adverse effects on patients.

Conclusion

SURE has the potential to be a useful clinical checklist for practitioners, responding to the growing need for detecting clinically significant decisional conflict in patients with a brief “practice friendly” instrument. It shows adequate psychometric properties in a primary care population with low prevalence of clinically significant decisional conflict. Although this needs further investigation, detecting clinically significant decisional conflict in patients could be used as a proxy to assess whether shared decision making has occurred, to evaluate the quality of the decision, and to ensure informed consent. Even if there is significant interest in decisional conflict capture, few studies have yet assessed the barriers and facilitators to routine use of an easy screening test such as SURE.¹⁶ Future studies should focus on identifying barriers and facilitators to the use of SURE in clinical practice and using these observations to design interventions that encourage health care providers to implement SURE in their daily practice. SURE’s internal consistency and diagnostic validity should also be tested in populations with high prevalence of clinically significant decisional conflict and in a variety of clinical contexts.

Footnotes

Acknowledgements

A.F.P. wrote the study protocol, established the a priori hypothesis, supervised the data analysis, interpreted the results, and wrote the first draft of this manuscript. F.L., M.L., M.R., and S.T. revised the manuscript critically for important intellectual content. A.F.P., F.L., M.L., M.R., and S.T. approved the final submitted version of the manuscript. A special acknowledgement to Dr. Brian Hess, who kindly accepted to review this manuscript and gave the authors valuable feedback. The authors would also like to acknowledge the generous contribution of the patients who participated in the study, and the entire recruitment team.

Fonds de recherche en santé du Québec (Quebec health research funding agency) and Conseil du médicament du Québec (Quebec medication board), now the Institut national d’excellence en santé et en services sociaux (national institute for excellence in health and social services).

References

Janis

Mann

. Decision making: A Psychological Analysis of Conflict, Choice and Commitment. New York: Free Press; 1977.

Carpentino-Moyet

, editor. Nursing Diagnosis: Application to Clinical Practice. 11th ed. Baltimore (MD): Lippincott Williams & Wilkins; 2006.

O’Connor

Jacobsen

. Conflit décisionnel:évaluation et aide apportée aux clients confrontés à des décisions concernant leur santé. Ottawa: Ottawa Hospital Research Center; 1998 [updated 2000]. Available from: URL: http://decisionaid.ohri.ca/francais/docs/Conflit_decisionnel.pdf

Brehaut

O’Connor

Wood

. Validation of a decision regret scale. Med Decis Making. 2003;23(4):281–92.

Gattellari

Ward

. Will men attribute fault to their GP for adverse effects arising from controversial screening tests? An Australian study using scenarios about PSA screening. J Med Screen. 2004;11(4):165–9.

Towle

Godolphin

. Framework for teaching and learning informed shared decision making. BMJ. 1999;319(7212):766–71.

Elwyn

Edwards

Britten

. “Doing prescribing”: how might clinicians work differently for better, safer care. Qual Saf Health Care. 2003 12 Suppl 1:i33–6.

Kryworuchko

Stacey

Bennett

Graham

. Appraisal of primary outcome measures used in trials of patient decision support. Patient Educ Couns. 2008;73(3):497–503.

O’Connor

. Validation of a decisional conflict scale. Med Decis Making. 1995;15(1):25–30.

10.

Lyerly

Nakagawa

Kuppermann

. Decisional conflict and the disposition of frozen embryos: implications for informed consent. Hum Reprod. 2011;26(3):646–54.

11.

Linder

Swank

Vernon

Mullen

Morgan

Volk

. Validity of a low literacy version of the Decisional Conflict Scale. Patient Educ Couns. 2011;85(3):521–4.

12.

Scholl

Loon

Sepucha

Elwyn

Legare

Harter

. Measurement of shared decision making—a review of instruments. Z Evid Fortbild Qual Gesundhwes. 2011;105(4):313–24.

13.

Simon

Loh

Harter

. Measuring (shared) decision-making—a review of psychometric instruments. Z Arztl Fortbild Qualitatssich. 2007;101(4):259–67.

14.

O’Connor

. User Manual—Decisional conflict Scale. Ottawa: Ottawa Hospital Research Center; 1993 [updated 2010]. Available from: URL: http://decisionaid.ohri.ca/docs/develop/User_Manuals/UM_Decisional_Conflict.pdf

15.

Légaré

Graham

O’Connor

Dolan

Bélanger-Ducharme

. Prise de décision partagée: traduction et validation d’une échelle de confort décisionnel du médecin. Pédagogie médicale. 2003;4:216–22.

16.

Legare

Graham

O’Connor

. Prediction of health professionals’ intention to screen for decisional conflict in clinical practice. Health Expect. 2007;10(4):364–79.

17.

Legare

Kearing

Clay

. Are you SURE? Assessing patient decisional conflict with a 4-item screening test. Can Fam Physician. 2010;56(8):e308–14.

18.

Jaeschke

Guyatt

Sackett

. Users’ guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA. 1994;271(9):703–7.

19.

Jaeschke

Guyatt

Sackett

. Users’ guides to the medical literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1994;271(5):389–91.

20.

Aertgeerts

Buntinx

Ansoms

Fevery

. Screening properties of questionnaires and laboratory tests for the detection of alcohol abuse or dependence in a general practice population. Br J Gen Pract. 2001;51(464):206–17.

21.

Aertgeerts

Buntinx

Kester

. The value of the CAGE in screening for alcohol abuse and alcohol dependence in general clinical populations: a diagnostic meta-analysis. J Clin Epidemiol. 2004;57(1):30–9.

22.

Cremonte

Ledesma

Cherpitel

Borges

. Psychometric properties of alcohol screening tests in the emergency department in Argentina, Mexico and the United States. Addict Behav. 2010;35(9):818–25.

23.

Legare

Labrecque

Godin

. Training family physicians and residents in family medicine in shared decision making to improve clinical decisions regarding the use of antibiotics for acute respiratory infections: protocol for a clustered randomized controlled trial. BMC Fam Pract. 2011;12:3.

24.

Legare

Labrecque

LeBlanc

. Training family physicians in shared decision making for the use of antibiotics for acute respiratory infections: a pilot clustered randomized controlled trial. Health Expect. 2011;14 Suppl 1:96–110.

25.

Legare

Labrecque

Cauchon

Castel

Turcotte

Grimshaw

. Training family physicians in shared decision-making to reduce the overuse of antibiotics in acute respiratory infections: a cluster randomized trial. CMAJ. 2012;184(13):E726–34.

26.

Fay

Proschan

. Wilcoxon-Mann-Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Stat Surv. 2010;4:1–39.

27.

Dawson

Weiss

. Dichotomizing continuous variables in statistical analysis: a practice to avoid. Med Decis Making. 2012;32(2):225–6.

28.

MacCallum

Zhang

Preacher

Rucker

. On the practice of dichotomization of quantitative variables. Psychol Methods. 2002;7(1):19–40.

29.

Naggara

Raymond

Guilbert

Roy

Weill

Altman

. Analysis by categorizing or dichotomizing continuous variables is inadvisable: an example from the natural history of unruptured aneurysms. AJNR Am J Neuroradiol. 2011;32(3):437–40.

30.

Balde

Legare

Labrecque

. Assessment of needs of men for decision support on male sterilization. Patient Educ Couns. 2006;63(3):301–7.

31.

Labrecque

Paunescu

Plesu

Stacey

Legare

. Evaluation of the effect of a patient decision aid about vasectomy on the decision-making process: a randomized trial. Contraception. 2010;82(6):556–62.

32.

Lalonde

O’Connor

Duguay

Brassard

Drake

Grover

. Evaluation of a decision aid and a personal risk profile in community pharmacy for patients considering options to improve cardiovascular health: the OPTIONS pilot study. IJPP. 2006;14:51–62.

33.

Patel

Wisner

. Decision making for depression treatment during pregnancy and the postpartum period. Depress Anxiety. 2011;28(7):589–95.

34.

O’Connor

Drake

Wells

Tugwell

Laupacis

Elmslie

. A survey of the decision-making needs of Canadians faced with complex health decisions. Health Expect. 2003;6(2):97–109.

35.

Sun

. Predicting Downstream Effects of High Decisional Conflict: Meta-analyses of the Decisional Conflict Scale. Ottawa (ON): University of Ottawa; 2004.

36.

Cherry

Hing

Woodwell

Rechtsteiner

. National Ambulatory Medical Care Survey: 2006 summary. Natl Health Stat Report. 2008;6(3):1–39.

37.

Smith

Schmidt

Allensworth-Davies

Saitz

. Primary care validation of a single-question alcohol screening test. J Gen Intern Med. 2009;24(7):783–8.

38.

RNAO. Decision Support for Adults Living with Chronic Kidney Disease. Toronto (ON): Registered Nurses’ Association of Ontario; 2009. p. 25–6.

39.

Murray

Thomas

Wald

Marticorena

Donnelly

Jeffs

. Exploring the impact of a decision support intervention on vascular access decisions in chronic hemodialysis patients: study protocol. BMC Nephrol. 2011;12:7.

40.

Stacey

Higuchi

Menard

Davies

Graham

O’Connor

. Integrating patient decision support in an undergraduate nursing curriculum: an implementation project. Int J Nurs Educ Scholarsh. 2009;6(1):Article10.

41.

Stacey

Samant

Pratt

Legare

. Feasibility of training oncology residents in shared decision making: a pilot study. J Cancer Educ. 2012;27(3):456–62.

42.

Morrow

Reed

Eliassen

Imset

. Shared decision making: skill acquisition for year III medical students. Fam Med. 2011;43(10):721–5.

43.

Stacey

Bennett

Barry

. Decision aids to help people who are facing health treatment or screening decisions. Cochrane Database Syst Rev. 2011;(10):CD001431.

44.

O’Connor

Fiset

DeGrasse

. Decision aids for patients considering options affecting cancer outcomes: evidence of efficacy and policy implications. J Natl Cancer Inst Monogr. 1999(25):67–80.

45.

Clark

O’Connor

Graham

Wells

. What factors are associated with a woman’s decision to take hormone replacement therapy? Evaluated in the context of a decision aid. Health Expect. 2003;6(2):110–7.

46.

Knops

Goossens

Ubbink

Legemate

Stalpers

Bossuyt

. Interpreting patient decisional conflict scores: behavior and emotions in decisions about treatment. Med Decis Making. 2013;33(1):78–84.