The Effect of Question Wording in Questionnaire Surveys on Placebo Use in Clinical Practice

Abstract

To identify factors that contribute to the high variability of the rates of use of placebo interventions reported in questionnaire surveys, the author investigated the effect of the explicit use of the word “placebo” in questionnaire surveys on placebo use in clinical practice on the results obtained. 190 primary care physicians in Poland were divided randomly into two groups. The groups received a questionnaire in which either the word placebo or the term “nonspecific methods of treatment” was used. The respondents who were asked explicitly about the use of placebo interventions declared that they never used placebo interventions significantly more often than participants asked about the use of nonspecific treatment methods. Moreover, the former reported significantly rarer use of placebo interventions than the latter. The study demonstrates that differences in the wording of questions in questionnaire surveys on placebo use can create statistically significant differences in results.

Keywords

placebo effect of question wording questionnaire surveys clinical practice primary care

Introduction

In a recent review, Fässler, Meissner, Schneider, and Linde (2010) identified 23 publications that presented the results of research aimed at defining the scope and frequency of placebo interventions by physicians and nurses, and also—much more rarely—by medical interns, medical students, and patients, as well as the circumstances that surrounded and attitudes toward such interventions (Berger, 1999; Bernateck et al., 2009; Berthelot, Maugars, Abgrall, & Prost, 2001; Chen & Johnson, 2009; Classen & Feingold, 1985; Comaroff, 1976; Ernst & Abbot, 1997; Fässler, Gnädinger, Rosemann, & Biller-Andorno, 2009; Goldberg, Leigh, & Quinlan, 1979; Goodwin, Goodwin, & Vogel, 1979; Gray & Flynn, 1981; Hróbjartsson & Norup, 2003; Lange, 1981; Lim & Seet, 2007; Lynöe, Mattsson, & Sandlund, 1993; Nitzan & Lichtenberg, 2004; Saupe, 1986; Schwartz, Soumerai, & Avorn, 1989; Shapiro & Struening, 1973a, 1973b; Sherman & Hickner, 2008; Thomson & Buchanan, 1982; Tilburt, Emanuel, Kaptchuk, Curlin, & Miller, 2008). These reports prove that placebo interventions are indeed used in clinical practice; however, the data on the frequency of their use are highly variable. In studies that were based on questionnaires (and also in those based on interviews), the usage of placebos was admitted by a proportion of physicians (from 17% to 99%; Bernateck et al., 2009; Classen & Feingold, 1985; Fässler et al., 2009; Goodwin et al., 1979; Gray & Flynn, 1981; Hróbjartsson & Norup, 2003; Lynöe et al., 1993; Nitzan & Lichtenberg, 2004; Sherman & Hickner, 2008; Tilburt et al., 2008) and nurses (from 51% to 100%; Bernateck et al., 2009; Ernst & Abbot, 1997; Goldberg et al., 1979; Goodwin et al., 1979; Gray & Flynn, 1981; Nitzan & Lichtenberg, 2004; Saupe, 1986). However, when the subjects were asked to record all the cases of placebo interventions that were given during a specified period of time, the rate of placebo interventions they reported ranged from only 0.3% to 5.1% (Goodwin et al., 1979; Lange, 1981).

The varying results of the research mentioned above provoked us to investigate why the findings were so diverse. Factors that will contribute to the variability in the results include the type of sample used (random, local, or convenience), the cohort of participants (specialists, general practitioners, or nurses), the setting of the study (hospital, general practice, and private practice), the response rate (from 36% to 100%), the number of subjects (from 37 to 679), and even the country in which the research was conducted. Moreover, the understanding of the notion of placebo that is held by both the participants and the researchers might also play a significant role in the diversity of results acquired in the studies. Fässler et al. (2010) emphasized that, among the studies in which a definition of placebo was presented to the participants, there were substantial differences in these definitions. In addition, there were differences among the definitions that could be inferred from the way the questions were formed, even though no definition was stated explicitly in the questionnaires.

The negative connotation of the notion of placebo is also significant because the use of placebo interventions in medical treatment is controversial legally, ethically, and professionally. This explains why the actual use of placebo might be underestimated when physicians are asked explicitly whether they use placebo interventions in their practice. This argument is corroborated by the results of research conducted by Tilburt, Emanuel, Kaptchuk, Curlin, and Miller (2008) who obtained a rate of use of placebo intervention that ranged from 55% to 80% in the same cohort of physician participants depending on the questions asked, and whether or not those questions included the phrase “placebo treatment.” When the subjects were asked how often they recommended treatment primarily to enhance patient expectation, only 20% chose the answer “never.” However, when they were asked to indicate which of several treatments they had used within the past year primary as “placebo treatment,” 45% answered that they did not prescribe placebo treatments. However, it should be noted that the study conducted by Tilburt et al. was not designed to evaluate the effect of the use of the word “placebo” in a questionnaire on the results obtained. In addition, the study was neither of an experimental type nor conducted using the between-group design. In their work, the items that concerned the use of placebo were included in a survey on complementary and alternative medicine, which might have influenced the results obtained significantly. As a consequence, it is difficult to consider the results to be conclusive to the problem of the effects of question wording on the results of questionnaire surveys on placebo use in clinical practice.

Given the shortcomings of the work described above, the goal of the study described herein was to determine the effect of the explicit use of the word placebo on the results of a survey on placebo use in clinical practice. The second goal of the research was to provide a preliminary estimate of the scope, frequency, and types of placebo interventions used in clinical practice by a sample of Polish physicians. Poland, which is one of the newest members of the North Atlantic Treaty Organization and the European Union, is a developing, postcommunist country that has undergone major social and economic reforms, including reform of the health care system. Given that previous studies were conducted only in developed countries, we wondered whether the same results would be obtained from a questionnaire survey on placebo use carried out in a developing country, namely Poland.

Given that researchers have devoted very little attention to the convictions of physicians and nurses about the factors that determine the effectiveness of placebo interventions, it was decided that the study presented herein would also address this question. The data that are available currently show that the majority of physicians and nurses (and also patients) believe that individuals who respond to placebos display specific characteristic personality traits (Berthelot et al., 2001; Chen & Johnson, 2009; Ernst & Abbot, 1997; Fässler et al., 2009; Goodwin et al., 1979; Gray & Flynn, 1981; Shapiro & Struening, 1973a, 1973b). However, the personality of placebo responders is only one potential factor that might influence the effectiveness of placebo interventions. Hence, the third goal of this study was to investigate the beliefs of physicians concerning the factors that determine the effectiveness of placebo interventions and to test whether their opinions varied depending on whether the word placebo was included in the questions.

Method

Participants

The participants in the study were 190 primary care physicians (general practitioners) mainly from southeastern Poland. Of these, 104 specialized in internal medicine, 41 in family medicine, and 58 in pediatrics, with 13 of the respondents listing two specialities. The study was of an individual character and was conducted by appropriately trained 26 interviewers, who were students of psychology participating in a course on the mechanisms of placebo effects. Physicians were asked personally by the interviewers to participate in the study, and if they agreed, they were given a hard copy of a questionnaire to fill in. All of the physicians who had agreed to participate in the study filled in the questionnaire.

The sample examined was of the convenience type, that is, participants were not selected randomly for the study, but the respondents were assigned randomly to groups. The participants were divided into two equal groups that both contained 95 physicians: the first group received the placebo questionnaire, and the second received the “nonspecific methods” questionnaire. More specifically, each interviewer distributed five placebo questionnaires and five nonspecific methods questionnaires among the participants alternately, that is, every odd respondent received placebo questionnaire and every even respondent received the nonspecific methods questionnaire. The independent variable was the type of questionnaire used (placebo vs. nonspecific methods), whereas the dependent variables were the answers to the individual questions. Randomization, manipulation of independent variables, control of other independent variables, and measurement of dependent variables gave the study an experimental design.

Questionnaire

The study used two versions of the same questionnaire: The only difference between the versions was the term used to define placebo interventions. In the placebo version of the questionnaire, the word placebo was used explicitly, whereas in the nonspecific methods questionnaire the term “nonspecific methods of treatment” was used. However, both terms were defined in exactly the same way: “For our purposes, we construe [placebo interventions/nonspecific methods] to be all the medical substances, practices, and procedures whose efficacy is difficult to prove scientifically, even though they might seem efficacious.¹ These may be pharmacological treatments of both inactive types (e.g., sugar pills and injections of saline), and active types, where the latter are used in cases in which - at least theoretically - they should have no impact on the symptoms of a patient (as with antibiotics in the treatment of viral illnesses, or vitamins taken for fatigue). Frequently, methods from natural and/or alternative medicine (e.g., homeopathy and certain physiotherapeutic procedures) are considered to be examples of [placebo interventions/nonspecific methods]: they happen to be effective, yet the mechanism of their operation is impossible to explain scientifically.”

The above definition was proposed in the absence of a standardized definition of placebo and the probable infeasibility of creating one (see Gøtzsche, 1994). Although it is based on scientific knowledge of placebo effects, it was prepared only for the purposes of this study and it is rather scientific for the general public. It reflects the wide understanding of placebo and is close to one of the most popular scientific definitions of placebo, which was developed by Arthur K. Shapiro (Shapiro & Shapiro, 1997). Although from a narrow view, a placebo is defined as an inert substance and it is only one of many nonspecific methods, a broad understanding of the term placebo makes it possible to consider the terms nonspecific methods and placebo synonymously (see “meaning” in Moerman, 2002, pp. 13–15, 133–135; and “placebo-related effects” in Benedetti, 2009, pp. 36–37). Even if one does not agree that the terms placebo and nonspecific methods are synonymous both were defined in exactly the same way to avoid any differences in understanding between them.

The questionnaire was composed of four basic multiple-choice questions. Respondents were asked about the frequency of their use of placebo interventions/nonspecific methods (closed question that allowed the choice of a single option), the types of placebo interventions/nonspecific methods used (closed question and multiple answers), the placebo intervention/nonspecific method used most frequently (closed question and single option), and the most important factor that influenced the effectiveness of placebo interventions/nonspecific methods (closed question and single option²).

Data Management and Analysis

Descriptive statistics and frequencies were used to examine the individual traits of physicians and to report behaviors and attitudes. The percentages are based on the actual number of respondents. Pearson’s χ² test was used to compare the distribution of results acquired in the two groups examined, whereas to measure the strength of correlation between the variables, the correlation coefficient C and the Φ value for 2 × 2 tables and Cramer’s V for larger tables were calculated. To test the significance of the differences between the percentages obtained, the two-tailed significance test on two structural indicators was used. All analyses were conducted with STATISTICA (Version 9.0).

Results

The mean age of the subjects was 47.3 (range 27–77; SD 11.1); 63 respondents (33%) were male and 127 (67%) were female. The average length of work experience was 21 years (range 1–50; SD 10.8), and the average declared number of patients seen weekly was 123.7 (range 5–600; SD 85.8). A total of 22 physicians worked in small towns with populations below 10,000 (12%), 56 (29%) in cities with populations that ranged from 10,000 to 100,000, 70 (37%) in cities with populations of 100,000 to 500,000, and 42 (22%) in cities with populations of over 500,000. There were no significant differences with respect to age, sex, specialization, work experience, number of patients seen weekly, or town population between the groups who filled in the placebo and the nonspecific methods questionnaires.

Statistically significant differences between the groups were seen with respect to the answer to the question about the frequency of placebo usage (Table 1). Respondents given the placebo questionnaire asserted that they never used placebo interventions more often than the nonspecific methods respondents (16% vs. 5%, Pearson’s χ² (1, N = 182) = 5.21, p = .02). Nevertheless, the strength of the correlation between the type of questionnaire received and the use or nonuse of placebo interventions was moderate (Φ = −0.17, correlation coefficient C = 0.17).

Table 1.

Reported Frequency of Use of Placebo Interventions by Questionnaire Type

	Placebo (N ^a = 89)	Nonspecific Methods (N ^a = 93)
Frequency	No. (%)	No. (%)
Every day	7 (8)	19 (20)
Once a week	16 (18)	38 (41)
Once a month	30 (34)	22 (24)
Once a year	22 (25)	9 (10)
Never	14 (16)	5 (5)

Note. ^a N reflects the actual number of participants who answered the question.

Respondents who filled in the placebo questionnaire declared that they used placebo interventions significantly less frequently than those who answered the nonspecific methods questionnaire (Pearson’s χ² (4, N = 182) = 25.37, p = .0004). Here, the connection between the type of the questionnaire received and the stated frequency of use of placebo interventions was clear (correlation coefficient C = 0.35, Cramer’s V = 0.37). The subjects given the nonspecific methods questionnaire reported that they used placebo interventions daily or once a week more frequently than those given the placebo version, whereas the placebo subjects declared that they used placebo interventions once a month or once a year more frequently than the other group.

The type of questionnaire was found to have no effect on the types of placebo interventions used or on the placebo intervention used most frequently (Table 2). Irrespective of the type of questionnaire completed, the participants stated that the following were the placebo interventions they used most frequently: vitamins (placebo respondents [P] 57%; nonspecific methods subjects [N] 54%); homeopathy (P 44% and N 47%); dietary supplements (P 42% and N 48%); and pharmacologically inactive substances (e.g., sugar pills and saline; P 37% and N 35%). With respect to the placebo intervention used most frequently, the majority of respondents to the placebo questionnaire stated vitamins (33%), whereas 22% said inactive pharmacological substances and 21% said homeopathy. Among the subjects who answered the nonspecific methods questionnaire, homeopathy (28%) was used most, followed by vitamins (25%). However, these differences were not statistically significant.

Table 2.

Types of Placebo Intervention Applied and Most Frequently Used Placebo Interventions by Questionnaire Type

	Types of Placebo Used^a		Most Frequently Used Placebo
	Placebo (N ^b = 81)	Nonspecific Methods (N ^b = 89)	Placebo (N ^b = 81)	Nonspecific Methods (N ^b = 85)
Types of Placebo Intervention	No. (%)	No. (%)	No. (%)	No. (%)
Inactive pharmacological substances	30 (37)	31 (35)	18 (22)	13 (15)
Vitamins	46 (57)	48 (54)	27 (33)	21 (25)
Dietary supplements	34 (42)	43 (48)	7 (9)	9 (11)
Homeopathy	36 (44)	42 (47)	17 (21)	24 (28)
Alternative medicine	10 (12)	11 (12)	3 (4)	1 (1)
Too small a dose of active substances	5 (6)	9 (10)	2 (2)	2 (2)
An active substance that brings no specific result in the given case	9 (11)	17 (19)	3 (4)	6 (7)
Practices or procedures that bring no specific result in the given case^c	11 (14)	13 (15)	3 (4)	7 (8)
Switched-off medical devices	2 (2)	3 (3)	1 (1)	—
Other^d	1 (1)	5 (6)	—	2 (2)

Note. ^aGiven that respondents were allowed to choose more than one answer, the percentages do not add up to 100.

^b N reflects the actual number of participants who answered the question, especially without those who declared that they never used placebo interventions.

^cAs examples of such methods, two respondents listed cupping glasses and one saline inhalations.

^dAmong other types of placebo intervention, two respondents mentioned cupping glasses and one simply talking to the patient.

Respondents who were given the placebo questionnaire (N = 85)³ indicated the individual traits of patients (including age, sex, and personality) most frequently as decisive factors in the effectiveness of placebo interventions (43, 51%). This was followed by patient expectations (29, 34%). Respondents to the nonspecific methods questionnaire (N = 90)³ gave very similar levels of importance to patient expectations (44, 49%) and the individual traits of patients (43, 48%) in determining the effectiveness of placebo interventions. There were statistically significant differences between the two groups of participants with respect to the frequency of selection of the answer “patient expectations.” The two-tailed significance test on two structural indicators (percentages) for the variable “patient expectations” showed that the participants who completed the nonspecific methods questionnaire listed patient expectations as the factor that influenced most the effectiveness of placebo interventions significantly more often than those who answered the placebo questionnaire (p = .048). Far fewer subjects listed factors on the physician’s side, namely, the traits of the physician (P 14, 16%; N 13, 14%) or their expectations (P 0, 0%; N 1, 1%), or the type of illness (P 10, 12%; N 15, 17%). Only one person chose the answer “other,” mentioning “time for the organism’s self-defense” (P 0, 0%; N 1, 1%).

Discussion

Summary of Principal Findings

The results of this study demonstrated that the way in which placebo is described in questionnaire surveys on its use in clinical practice influences the results obtained significantly. Participants who were asked explicitly about the use of placebo interventions declared significantly more frequently that they never resorted to placebo interventions. Moreover, they reported significantly rarer use of placebo interventions than participants who were asked about the use of nonspecific treatment methods. Thus, the results of the study demonstrate that differences in the wording of questions in questionnaire surveys on placebo use can create statistically significant differences in results and suggest strongly that the high frequency of placebo use that is reported in questionnaire surveys is actually an underestimate. However, no effects of question wording was found when participants answered questions about the types of placebo/nonspecific methods used and the types of intervention used most frequently. Among the placebo interventions used, the participants listed vitamins, homeopathy, dietary supplements, and substances with no pharmacological effect (e.g., sugar pills and saline) most frequently.

The way in which placebo interventions were described also influenced the factors that the participants identified to be decisive in the effectiveness of placebo interventions. Irrespective of how placebo interventions were described, the responding physicians identified the individual traits of patients (e.g., age, sex, and personality) most frequently as decisive factors in the effectiveness of placebo interventions. However, whereas those responding to the nonspecific methods questionnaire listed patient expectations equally as often as patient traits, those given the placebo questionnaire gave this answer significantly less often than patient traits.

Strengths and Weaknesses of the Study

The strengths of this study include its novelty (in investigating the effect of the explicit use of the word placebo in questionnaire surveys on placebo use in clinical practice on the results obtained), the experimental design, random assignment to experimental groups, and comparability of the groups. However, some weaknesses of the study should also be acknowledged. There is no certainty that the respondents familiarized themselves fully with the definition of placebo interventions/nonspecific methods of treatment with which they were presented, and even if they did, whether this definition was in-line with their beliefs. Even though the definition was printed in bold type, it was relatively long, which might have discouraged its full comprehension. Even so, given the random assignment of participants to experimental groups, this limitation should not have biased the results obtained in the two groups.

There are more limitations for the results for the estimation of the behaviors and attitudes of physicians with respect to the use of placebo interventions in clinical practice. Given the convenience sample studied, the results obtained with respect to the behaviors and attitudes of Polish physicians concerning the use of placebo interventions in clinical practice might not be representative of the population of primary care/general practitioners in Poland, or Polish physicians in general. However, in general, the findings are consistent with the results of other published questionnaire surveys on placebo use in clinical practice.

Another limitation of the research is the self-reported retrospective outcome: It might have been difficult for the respondents to estimate the frequency of a particular behavior accurately. Although it is difficult for an individual to misremember whether they have ever used placebo interventions, they might misjudge the frequency of use. Nevertheless, the frequencies of use recorded in this study are among the highest published so far, and therefore the probability of underestimation is low.

Relationship to Other Studies

The results of the research presented herein provide experimental corroboration of the results of the survey on placebo use in clinical practice that was conducted by Tilburt et al. (2008). These authors obtained two different rates of placebo intervention (55% and 80%) within the same sample of physicians, depending on whether or not the words placebo treatment were included in the questions. Owing to the use of an experimental design in this study, it was possible to demonstrate the influence of the way in which placebo interventions were described on the results of questionnaires, with full control of other variables. Moreover, the description used did not only affect the reported rate of application of placebo interventions but also the reported frequency of their application and the stated beliefs of the participants about the influence of patient expectations on the effectiveness of placebo interventions. These findings were consistent with the results of research that showed that differences in the wording of questions in questionnaire surveys can create statistically significant differences in results (Brener, Grunbaum, Kann, McManus, & Ross, 2004; Wahl, Svensson, & Hydén, 2010).

The study described herein is the first questionnaire survey on placebo use in clinical practice conducted in Poland. The reported frequency of use of placebo interventions among the group who responded to the placebo questionnaire was nearly identical in this study to that obtained in a study by Hróbjartsson and Norup (2003; 84% vs. 86%). The earlier study was conducted on a sample of general practitioners in Denmark, a country not very distant from Poland. However, these two rates are among the highest obtained so far in studies of groups of physicians. The frequency of use of placebo intervention reported in this study was also among the highest yet reported, but not the very highest. In the group that was asked explicitly about the use of placebo interventions, 60% of respondents declared at least monthly use, whereas in American studies, the corresponding rates were 80% (Tilburt et al., 2008) and 8% (Sherman & Hickner, 2008), and in German studies, 70% (Classen & Feingold, 1985) and 13% (Bernateck et al., 2009).

Irrespective of how placebo interventions were described, the physicians who participated in the study reported patient traits (e.g., age, gender, and personality) most frequently to be the factor that determines the effectiveness of these interventions. This finding corroborates the results of earlier studies (Berthelot et al., 2001; Chen & Johnson, 2009; Ernst & Abbot, 1997; Fässler et al., 2009; Goodwin et al., 1979; Gray & Flynn, 1981; Shapiro & Struening, 1973a, 1973b). However, unlike the majority of studies conducted so far, the current study also gave the option of other factors among the possible answers to the question about effectiveness. A large proportion of the participants stated that patient expectations were a decisive factor in the effectiveness of placebo interventions, whereas far fewer participants listed factors on the physician’s side, namely, the traits of the physician and their expectations, or the type of illness. It is interesting that hardly any of the physicians who responded to the survey believed that their own expectations influenced the effectiveness of placebo interventions, although convincing evidence for the role of the expectations of the physician in the placebo effect does exist (Gracely, Dubner, Deeter, & Wolskee, 1985).

Conclusions

Questionnaire surveys on the use of placebo in clinical practice are susceptible to the effect of question wording. The way in which placebo interventions are described influences the results obtained. The explicit use of the term placebo decreased the number of participants who declared the use of placebo interventions in clinical practice significantly and lowered the declared frequency of use of placebo interventions by those who used them. These results suggest strongly that the high frequency of placebo use reported in questionnaire surveys is actually an underestimate. Irrespective of how placebo interventions are described, the results of the study described herein suggest that their application is very common among Polish doctors in primary care. Nevertheless, to describe the behaviors and attitudes of Polish physicians with respect to the use of placebo interventions in clinical practice more fully, studies should be conducted on a representative random sample.

Footnotes

Acknowledgments

The author would like to thank the students of psychology at the Institute of Psychology, Jagiellonian University who participated in the course Mechanisms of Placebo Effects, conducted in the summer semester 2010, for their assistance in conducting the research.

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Benedetti

(2009). . In Placebo effects: Understanding the mechanisms in health and disease New York, NY: Oxford University Press.

Berger

J. T.

(1999). Placebo medication use in patient care: A survey of medical interns. Western Journal of Medicine, 170, 93–96.

Bernateck

Karst

Eberhard

Vivell

Fischer

M. J.

Stichtenoth

D. O.

(2009). Placebotherapie. Analyse von Umfang und Erwartung in einer Klinik der Maximalversorgung. Schmerz, 23, 47–53.

Berthelot

J. M.

Maugars

Abgrall

Prost

(2001). Interindividual variations in beliefs about the placebo effect: A study in 300 rheumatology inpatients and 100 nurses. Joint Bone Spine: Revue du Rhumatisme, 68, 65–70.

Brener

N. D.

Grunbaum

Kann

McManus

Ross

(2004). Assessing health risk behaviors among adolescents: The effect of question wording and appeals for honesty. Journal of Adolescent Health, 35, 91–100.

Chen

G. F.

Johnson

M. H.

(2009). Patients' attitudes to the use of placebo: Results from a New Zealand survey. New Zealand Medical Journal, 122, 35–46.

Classen

Feingold

(1985). Use of placebos in medical practice. Pharmacopsychiatry, 18, 131–132.

Comaroff

(1976). A bitter pill to swallow: Placebo therapy in general practice. Sociological Review, 24, 79–96.

Ernst

Abbot

N. C.

(1997). Placebos in clinical practice: Results of a survey of nurses. Perfusion, 10, 128–130.

10.

Fässler

Gnädinger

Rosemann

Biller-Andorno

(2009). Use of placebo interventions among swiss primary care providers. BMC Health Services Research, 9, 144.

11.

Fässler

Meissner

Schneider

Linde

(2010). Frequency and circumstances of placebo use in clinical practice–a systematic review of empirical studies. BMC Medicine, 8, 15.

12.

Goldberg

R. J.

Leigh

Quinlan

(1979). The current status of placebo in hospital practice. General Hospital Psychiatry, 1, 196–201.

13.

Goodwin

J. S.

Goodwin

J. M.

Vogel

A. V.

(1979). Knowledge and use of placebos by house officers and nurses. Annals of Internal Medicine, 91, 106–110.

14.

Gøtzsche

P. C.

(1994). Is there logic in the placebo?. Lancet, 344, 925–926.

15.

Gracely

Dubner

Deeter

Wolskee

(1985). Clinicians' expectations influence placebo analgesia. Lancet, 325, 43.

16.

Gray

Flynn

(1981). A survey of placebo use in a general hospital. General Hospital Psychiatry, 3, 199–203.

17.

Hróbjartsson

Norup

(2003). The use of placebo interventions in medical practice—A national questionnaire survey of Danish clinicians. Evaluation & the Health Professions, 26, 153–165.

18.

Lange

H. U.

(1981). Zur Praxis der Placebobehandlung in einer psychiatrischen Klinik. Psychiatrische Praxis, 8, 147–151.

19.

Lim

E. C. H.

Seet

R. C. S.

(2007). Attitudes of medical students to placebo therapy. Internal Medicine Journal, 37, 156–160.

20.

Lynöe

Mattsson

Sandlund

(1993). The attitudes of patients and physicians toward placebo treatment: A comparative study. Social Science & Medicine, 36, 767–774.

21.

Moerman

D. E.

(2002). . In Meaning, Medicine and the “Placebo Effect.” London, England: Cambridge University Press.

22.

Nitzan

Lichtenberg

(2004). Questionnaire survey on use of placebo. British Medical Journal, 329, 944–946.

23.

Saupe

(1986). The use of placebo as seen by nursing staff. Pharmacopsychiatry, 19, 347–348.

24.

Schwartz

R. K.

Soumerai

S. B.

Avorn

(1989). Physician motivation for nonscientific drug prescribing. Social Science & Medicine, 28, 577–582.

25.

Shapiro

A. K.

Shapiro

(1997). . In The powerful placebo: From ancient priest to modern physician Baltimore, MD: Johns Hopkins University Press.

26.

Shapiro

A. K.

Struening

E. L.

(1973a). Defensiveness in the definition of placebo. Comprehensive Psychiatry, 14, 107–120.

27.

Shapiro

A. K.

Struening

E. L.

(1973b). The use of placebos: A study of ethics and physicians' attitudes. Psychiatry in Medicine, 4, 17–29.

28.

Sherman

Hickner

(2008). Academic physicians use placebos in clinical practice and believe in the mind-body connection. Journal of General Internal Medicine, 23, 7–10.

29.

Thomson

R. J.

Buchanan

W. J.

(1982). Placebos and general practice: Attitudes to, and use of the placebo effect. New Zealand Medical Journal, 95, 492–494.

30.

Tilburt

J. C.

Emanuel

E. J.

Kaptchuk

T. J.

Curlin

F. A.

Miller

F. G.

(2008). Prescribing ‘placebo treatments': Results of national survey of US internists and rheumatologists. British Medical Journal, 337, 1097–1100.

31.

Wahl

Svensson

Å.

Hydén

(2010). Effects of minor phrasing variations in traffic-related questionnaires—Comparison of objective equivalences and respondents' subjective statements. Transportation Research: Part F, 13, 315–328.