A Validated Questionnaire to Assess the Knowledge of Psychiatric Aspects of Alcohol Use Disorder

Abstract

Background

To the best of our knowledge, there is no validated instrument for measuring knowledge of psychiatric aspects of alcohol use disorder (AUD) amongst medical students. Our aim was to develop an instrument for this purpose and to describe the instrument's psychometric properties. We also investigated whether the instrument could demonstrate a significant change in scores following an educational intervention consisting of a 60-minute PowerPoint lecture on AUD, associated handouts, and role-plays.

Methods

The Knowledge of Psychiatric Aspects of Alcohol Questionnaire (KPAAQ) was developed from the Kaplan and Saddock textbook synopsis chapter on alcohol related disorders. The questionnaire included 6 categories of clinically relevant material: metabolism of alcohol, short-term effects of alcohol, long-term effects of alcohol, AUD, alcohol withdrawal, and alcohol use in pregnancy. The KPAAQ was administered to 75 medical students in Years 4 and 5 from the Hebrew University in Jerusalem, Israel, during a relevant clinical rotation. Following the initial administration of the KPAAQ, the students attended a 60-minute lecture (in Hebrew) based on material from the University of Sydney's learning module on alcohol. The KPAAQ was readministered to the students immediately following the lecture.

Results

The KPAAQ demonstrated good reliability (Cronbach α =.92 for all questions) and validity r(209) =.674, P <.001. Knowledge of alcohol and alcoholism significantly increased after the educational intervention (F(2, 154) = 151.60, P <.001). Post hoc comparisons using the Scheffe test revealed a significant positive change in knowledge after students received the intervention (mean difference = 33, P <.001).

Conclusions

These initial findings suggest that the KPAAQ is a reliable and valid instrument for assessing medical student knowledge of psychiatric aspects of AUD over 6 clinical categories.

Keywords

Alcoholism dual-diagnosis psychiatry education

Introduction

A substantial proportion of alcoholics seek medical treatment for their physical complications rather than seek treatment for alcoholism.⁽¹⁾ Referral to an emergency department or admission to a general hospital, therefore, provides a window of opportunity for intervention for these patients.⁽²⁾ Regrettably, however, there is evidence that doctors are not effective in detecting alcohol use disorder (AUD) in the hospital setting and elsewhere.⁽1^,3⁾ Lack of appropriate training has been highlighted as a critical barrier for medical practitioners confronted by patients with alcohol problems.⁽4^,5⁾ The National Institute on Alcohol Abuse and Alcoholism(6) has recommended that the impact of changing the medical curriculum should be examined with experimental or quasi-experimental designs. The International Centre for Drug Policy described the core aims and learning outcomes in medical undergraduate curricula and good practice on delivery.⁽⁷⁾ A number of interventions to improve detection rates of AUD in the hospital have been described, including a single screening question,⁽⁸⁾ individual feedback to junior medical officers,3 and the establishment of a Drug and Alcohol Unit with a drug and alcohol education program.⁽⁹⁾ Medical students have been surveyed about their knowledge, skills, and attitudes regarding alcohol and drugs, but the validity and reliability of the survey instrument is unclear.⁽10 ^–14⁾ Low levels of knowledge were demonstrated in the areas of psychiatric complications of alcohol abuse, screening and low-risk drinking guidelines, problem drinking, and physician impairment.⁽¹⁴⁾

In the Kahan et al. study,⁽¹⁴⁾ 26 multiple choice questions about alcohol and drugs were drafted using learning objectives from a review of medical curricula by medical faculty staff at Ontario's 5 medical schools. The Roche et al. study⁽¹¹⁾ utilized a survey of 120 items about alcohol and drugs, which was developed through consultation with staff of the Family Medicine Program, drug and alcohol specialists, and medical educators. The survey used a variety of question types, including multiple choice, true/false, and case vignettes, and it was estimated to take about 30 minutes to complete. No reliability or validity properties were reported for the instruments used in these 2 studies.

The Students Alcohol Test⁽¹⁵⁾ (SAQ) is a validated and reliable questionnaire that examines students’ behaviors and knowledge regarding alcohol. It contains 4 subscales relating to patterns of alcohol use, knowledge about alcohol, knowledge about problems resulting from alcohol, and attitudes towards alcohol. This tool utilizes a true/false/don't know format and encourages the student not to guess if he/she does not know the answer to a question. This format provides a valuable means of identifying deficiencies in the student's knowledge base and facilitates a process of monitoring subsequent progress in learning the relevant material.

There is little research that addresses the baseline knowledge of the medical student in this area and the retention of this knowledge following an educational intervention. Alcohol withdrawal syndromes in the general hospital⁽²⁾ and AUD in pregnancy⁽¹⁶⁾ have been identified as areas in which doctors need more training in recognition and assessment. The limited awareness of AUD by medical staff may relate to the view that alcohol morbidity has not been a serious problem in certain cultures.⁽¹⁾

It was felt that a valid and reliable questionnaire for testing medical students’ knowledge of alcohol in clinical and nonclinical domains was required to provide a tool that could be used to identify areas of deficient knowledge and to demonstrate that these deficiencies have been rectified following a teaching intervention.

In this paper, the development of a Knowledge of Psychiatric Aspects of Alcohol Questionnaire (KPAAQ) for testing medical students’ knowledge about AUD will be described, including the instrument's psychometric properties. The KPAAQ was also used to examine whether significant changes in students’ knowledge base could be achieved following an intervention (60-minute PowerPoint lecture on AUD, directed student role-play, and contact with a representative of Alcoholics Anonymous.).

Methods

The study was undertaken at the Shaare Zedek Medical Centre, which is affiliated with the Hebrew University, Jerusalem, following approval by the hospital's ethics committee. This study conforms to the provisions of the Declaration of Helsinki.

The Students Alcohol Test (SAQ) was chosen as a basis for the KPAAQ because the scoring format facilitates identification of the student's deficient knowledge base. The SAQ is dedicated to alcohol-related issues, has robust psychometric properties, and enjoys widespread use over the last 40 years. Its reliability has been updated more recently.⁽¹⁷⁾ Twenty questions from the SAQ knowledge of alcohol subscale (consisting of 35 questions) were adapted as a basis for the KPAAQ. Items were selected from the SAQ on the basis of relevance to taking a history from a person with AUD. Items that had specific cultural and historical associations (such as references to the American Puritan Movement) were not included. Since this scale does not contain clinical material relating to alcohol withdrawal syndromes, a further 30 questions were adapted from the Kaplan and Saddock textbook synopsis chapter on alcohol related disorders⁽¹⁸⁾ in order to supplement the KPAAQ. The process of item selection and wording was reviewed by an expert panel of medical clinicians for relevance and validity. The questionnaire was translated from English into Hebrew by one of the authors (S.J.). The translation back from Hebrew and into English for comparison was validated by another author (C.G.). (See Appendix for the complete questionnaire.)

It was felt that medical students should be acquainted with this clinical material so that, as doctors, they would be able to diagnose and assess these conditions and refer as appropriate for further treatment. The questionnaire was divided into 6 categories of clinically relevant material related to alcohol use and misuse: metabolism of alcohol, short-term effects of alcohol, long-term effects of alcohol, AUD, alcohol withdrawal, and alcohol use in pregnancy. The scoring format allows the respondent to acknowledge areas of deficient knowledge without having to guess as a default option. This is a significant advantage because the deficiencies of any curriculum are best identified when a student is able to indicate that he or she does not know the material. This may direct teaching staff to focus on topics that are poorly understood and perhaps present material in a more effective fashion.

Seventy-five medical students from the Hebrew University Medical School (Years 4 and 5) agreed to participate in the study. The fourth-year students were doing a rotation in internal medicine, and the fifth-year students were doing a rotation in psychiatry. In their second year of training, the medical students had participated in a behavioral sciences lecture on alcohol abuse, which included material on the CAGE questionnaire.⁽¹⁹⁾ In their fifth year of training, the students were taught about fetal alcohol syndrome in their clinical obstetrics and gynecology rotation.

Table 1

Cronbach Alpha Coefficients of Questions for Each Category of the KPAAQ

Category	Question numbers	Cronbach alpha coefficient
Metabolism of alcohol	1, 8, 10, 11, 15, 17, 21, 23, 30, 32	.71
Short-term consequences of alcohol use	3, 5, 9, 13, 14, 19, 20, 22, 24, 25, 33, 43	.60
Long-term effects of alcohol use	6, 26, 27, 41	.70
Alcohol use disorder	4, 7, 12, 16, 18, 28, 29, 31, 44	.70
Alcohol withdrawal	2, 34, 35, 36, 37, 38, 39, 40, 42, 45, 46	.85
Alcohol use in pregnancy	47, 48, 49, 50	.42
	All questions	.92

The students were requested to complete the KPAAQ as part of a research project to determine their knowledge regarding psychiatric aspects of alcohol. Students completed the questionnaire anonymously, indicating their date of birth, year of medical training, and whether they had participated in a lecture on alcohol. In keeping with the instructions of the SAQ, the students were asked to indicate whether the statements in the questionnaire were true or false. They were requested not to guess the answers, but rather, to indicate if they did not know whether a statement was true or false. One of the authors (S.J.) delivered a 1-hour PowerPoint-assisted lecture on the psychiatric aspects of alcohol abuse to small groups of students of 12 or less and distributed material on identifying AUD (Alcohol Use Disorder Inventory [AUDIT] and Michigan Alcohol Screening Test—Geriatric Version [MAST-G]).

In order to reinforce the learning process, a member of Alcoholics Anonymous, who was experienced in public speaking, attended the lecture and provided the students with a personal account of his alcohol addiction, and wherever possible, made a connection between his illness experience and the lecture material. For example, the speaker described the delay in making a diagnosis of AUD by his general practitioner despite repeated requests for sick certificates.

In addition, some of the students were involved in structured role-play scenarios that entailed taking an alcohol and substance abuse history, completing an AUDIT, communicating with a young person about the personal and social pressures associated with alcohol use disorder, the dangers of alcohol in pregnancy, and how to talk to a family about AUD.

The students completed the KPAAQ immediately after the lecture. The KPAAQ could be completed by most students in 10 to 15 minutes. There was a good completion rate between prelecture and postlecture completion of KPAAQ (67 students out of the 75 students completed the KPAAQ after the lecture: 89% of the original sample). All measurements were accomplished in person.

Statistical Analysis

In order to determine the reliability of the questionnaire, a Kuder-Richardson analysis was performed for internal consistency of each category (metabolism of alcohol, short-term effects of alcohol, long-term effects of alcohol, alcohol use disorder, alcohol withdrawal, and alcohol use in pregnancy) and also for the whole questionnaire, yielding Cronbach alpha values.⁽²⁰⁾ The KPAAQ was completed anonymously, so there was no way of identifying paired results prior to and following the educational intervention. Not all participants completed the KPAAQ after the intervention.

The questionnaire's validity was determined using a linear correlation between the results of the 20 questions based on the validated SAQ and the remaining 30 questions of our questionnaire. Two Pearson correlations were done: one for the “before” scores and one for the “after” scores. A simple t test was conducted to compare the student's knowledge of alcohol and alcoholism before and after the intervention. Another simple t test was conducted to compare the number of “don't know” responses before and after the intervention.

Results

Each response to the 50 questions of the KPAAQ was assessed as to whether it was correct or not. Each correct response was awarded 2 points, thereby, yielding a possible range from 0 to 100 points.

A Cronbach alpha was calculated for each of the 6 categories of the KPAAQ and for the questionnaire as a whole in order to determine the internal consistency of these measures (see Table 1). The internal consistency of the KPAAQ was found to be good (Cronbach α =.92) and a reasonable internal consistency was found for all of the categories (.85 > α >.60) except for the category addressing alcohol use in pregnancy (Cronbach α =.42).

In order to determine the validity of the KPAAQ compared with the preexisting SAQ, a Pearson correlation was performed between 20 questions from the SAQ and the remaining 30 items of the KPAAQ, and a reasonable correlation was found (r(67) =.67). Since the KPAAQ was developed in order to expand the clinical use of the SAQ rather than as a replacement for it, this level of correlation was considered to be good. It was decided not to identify the expected performance of the SAQ.

A t test was conducted to compare the student's knowledge on alcohol and alcoholism before the intervention and immediately after the intervention (see Table 2). There was a significant difference in knowledge of alcohol and alcoholism (t(140) = 16.8, P <.001). Another simple t test was conducted to compare the number of “don't know” responses before and after the intervention. There was a significant decrease in the numbers of “don't know” responses on the KPAAQ after the intervention compared with before the intervention (t(140) = 15.4, P <.001).

Table 2

Average Knowledge of Alcohol and Alcoholism Before and After Intervention

Item	Before (n = 75)	After (n = 67)	t(140)	P
Knowledge (M ± SD)	44 ± 13	77 ± 10	16.8	<.001
“Don't know” responses (M ± SD)	23 ± 8	5 ± 5	15.5	<.001

Discussion

The KPAAQ was found to have good internal consistency, and the newly developed items of the instrument correlated with items from a known scale (SAQ). The sample size used in the research was sufficient to validate the KPAAQ relative to the SAQ and to yield Cronbach alpha scores. A possible explanation for the low alpha in the category “alcohol use in pregnancy” was the low number of question items (n = 4) representing this category. To the best of our knowledge, the KPAAQ is the only validated questionnaire for assessing medical students’ knowledge of psychiatric aspects of alcohol use.

Limitations

Weaknesses of the study include the potential differences in didactic teaching regarding alcohol and differences in clinical exposure between the fourth- and fifth-year medical students who participated in the research. Similarly, some of the teaching sessions did include role-play scenarios whereas others did not. However, our results indicate that there is good internal consistency in the properties of the KPAAQ.

Impacts of KPAAQ

Teaching strategies that engage the learner have been shown to improve the attitudes of medical students towards substance misusers⁽21 ^–23⁾ and facilitate the ability to incorporate this knowledge with previous experience and understanding.⁽24^,25⁾ Although the provision of small-group learning experiences and clinical placements is time-consuming, such educational experiences have been shown to have the potential to positively influence students’ knowledge, skills, attitudes, and confidence relating to substance misuse.⁽26 ^–29⁾ The KPAAQ could be completed by most students in 10 to 15 minutes. It can, therefore, be used as a simple device to help both the faculty staff and students to monitor progress in learning and retaining knowledge about the subject. We believe that the KPAAQ will significantly contribute towards improving the knowledge base of AUD amongst medical students and encourage the development of necessary skills for managing patients with AUD.⁽⁷⁾

Future Research

A strength of this study is that, as a relatively simple, reliable, and valid tool for measuring students’ knowledge of alcohol, the KPAAQ can be tested in other clinician populations who have contact with persons suffering from AUD. We plan to follow up with the medical students in their course, assess their retention of the material using the questionnaire, and also to compare them with hospital medical staff. We will assess doctors who work in a general hospital emergency department, doctors in the internal medicine and surgery departments, and, also, senior doctors with teaching responsibilities. Additionally, we intend to administer the KPAAQ to nonmedical staff who work with AUD patients. It is anticipated that these questions will assist medical practitioners in community and hospital settings to diagnose AUD in their patients, refer them to specialist alcohol services treatment programs, and collaborate in treatment as necessary. We are hopeful that this process will raise the awareness of AUD amongst hospital clinicians so that all postgraduate specialist trainees can improve their knowledge, skills, and attitudes towards patients with substance misuse in accordance with the recommendations of the working group of the UK medical royal colleges.⁽30^,31⁾

Acknowledgements

The authors would like to acknowledge the assistance of Prof. Amos Yinnon in planning the research, the support of Prof. Ruth Engs, the secretarial help provided by Brachi Jacobson and Yedida Ponger, and the continuing generosity of Yoram.

Author Contributions

Dr. Sol Jaworowski was involved in the conception, design and coordination of the research, the development of the KPAAQ, the translation of the KPAAQ from English to Hebrew, the coordination of KPAAQ allocation to the medical students, the delivery of the lecture and role play to the students, interpretation of the data, and drafting, critical revision and final approval of the article. Prof. Garry Walter was involved in the conception, design and coordination of the study, development of the KPAAQ, interpretation of the data, and drafting, critical revision and final approval of the article. Dr. Nerissa Soh was involved in the study design, statistical analysis, and drafting, critical revision and final approval of the article. Yossi Freier-Dror was responsible for the statistical analysis of the research. Dr. Joseph Mergui was involved in the design of the research and assisted in the allocation of the KPAAQ to the students. Dr. Cornelius Gropp was involved in the design of the research and in the translation of the KPAAQ from Hebrew to English. Prof. Paul Haber was involved in the conception, design and coordination of the research, the development of the KPAAQ and in the development of the lecture to the medical students, interpretation of the data, and drafting, critical revision and final approval of the article.

Footnotes

Appendix Knowledge of Psychiatric Aspects of Alcohol Questionnaire (Kpaaq)

WE WOULD LIKE TO ASK YOU FOR SOME INFORMATION ABOUT ALCOHOL.

The questions will either be True or False.

If you do not know the answer to the question, DO NOT GUESS. Mark a “0” in the box.

If you think the answer is TRUE write “1” for true.

If you think the answer is FALSE write “2” for false. 1.

Drinking milk before drinking an alcoholic beverage will slow the absorption of alcohol into the body. (SAQ)

Naltrexone is used as a medical treatment for alcohol abuse.

Alcoholic beverages do not provide weight-increasing calories. (SAQ)

Binge drinking of alcoholic beverages is more common in older people.

Alcohol is usually classified as a stimulant. (SAQ)

Alcohol is not an addictive drug. (SAQ)

The AUDIT questionnaire is a test for alcohol use disorder.

GGT (Gamma Glutamyl Transpeptidase) is the most sensitive enzyme available to detect alcohol abuse.

Many people drink to escape from problems, loneliness and depression. (SAQ)

10.

Alcoholic drinks mixed with water will affect you faster than alcohol drunk straight. (SAQ)

11.

A person weighing 70 kg, to keep his blood alcohol concentration below the legally intoxicated level, would have to drink fewer than 3 beers in an hour. (SAQ)

12.

A person cannot become an alcoholic by just drinking beer. (SAQ)

13.

To prevent getting a hangover, one should sip one's drink slowly, drink and eat at the same time, space drinks over a period of time, and not drink over one's limit. (SAQ)

14.

Drinking in moderation can result in relaxation, enhanced social interactions, and a feeling of well-being. (SAQ)

15.

Spirit drinks (whiskey, gin, vodka, etc.) usually contain about 15% alcohol by volume. (SAQ)

16.

Self help groups (Alcoholics Anonymous) are not helpful for those suffering from AUD.

17.

It takes about as many hours as the number of beers drunk to completely burn up the alcohol ingested. (SAQ)

18.

About 30% of people suffering from alcohol abuse suffer from a mood disorder during their lifetime. (SAQ)

19.

A blood alcohol concentration of.02% causes a person to be in a stupor. (SAQ)

20.

A glass of beer has very few calories so it has no impact on a diet. (SAQ)

21.

Proof on a bottle of alcoholic drink represents approximately half the percent of alcohol contained in the bottle. (SAQ)

22.

Beer usually contains from 2–5% alcohol by volume. (SAQ)

23.

Eating while drinking will slow down the absorption of alcohol into the body. (SAQ)

24.

Drinking coffee or taking a cold shower can be an effective way of sobering up. (SAQ)

25.

Consuming alcoholic drinks mixed with water is a way of avoiding getting drunk.

26.

Alcohol use is associated with about 50% of homicides and 25% of suicides.

27.

Alcohol abuse reduces life expectancy by about 10 years.

28.

Alcohol abuse is most prevalent among 18–29 year olds.

29.

Men are more likely than women to be binge drinkers.

30.

In an average sized person the amount of alcohol consumed from one standard drink is metabolized in 20 minutes. (SAQ)

31.

Four standard drinks per day is a moderate alcohol consumption for women.

32.

Women have higher levels of Alcohol dehydrogenase than men.

33.

Hypoglycaemia may be caused by acute alcohol intoxication.

34.

Autonomic hyperactivity is a feature of delirium tremens (DT's).

35.

DT's are less likely when the person enjoys good physical health.

36.

DT's usually commence 12–24 hours after stopping heavy alcohol intake.

37.

Vitamin B1 is effective in preventing DT's.

38.

Benzodiazepines are effective in preventing DT's.

39.

DT's normally appears after at least 5 years of heavy alcohol use.

40.

Epileptic convulsions are not seen in alcohol with-drawal.

41.

Thiamine is used to prevent the development of Korsakoff's Amnestic Syndrome.

42.

The mortality rate for untreated DT's approaches 15%.

43.

Alcohol use rarely causes disturbed sleep.

44.

There is no genetic basis to alcohol use disorders.

45.

Alcohol withdrawal-related hallucinations cannot be differentiated from DT's.

46.

Withdrawal from alcohol is more physically dangerous than withdrawal from heroin.

47.

Methadone is more dangerous than alcohol for a pregnant woman.

48.

Alcohol consumption in pregnancy does not affect the fetus.

49.

Alcohol consumption during pregnancy does not affect the child's post natal development.

50.

Microcephaly, craniofacial malformations and heart defects are commonly seen in infants affected with fetal alcohol syndrome.

Thank you for your cooperation.

(SAQ: Questions based on the Student Alcohol Questionnaire)

References

Chen

, Chen

, Cheng

ATA

. Prevalence and identification of alcohol use disorders among non psychiatric patients in one general hospital. Gen Hosp Psychiatry. 2004;26:219–225.

Jaworowski

, Raveh

, Golmard

, Gropp

, Mergui

. Alcohol use disorder in an Israeli general hospital: sociodemographic, ethnic and clinical characteristics. Isr Med Assoc J. 2012;14:294–298.

Proude

, Conigrave

, Britton

, Haber

. Improving alcohol and tobacco history taking by junior medical officers. Alcohol Alcohol. 2008;43:320–325.

Fassler

Views of medical students and residents on education in alcohol and drug abuse. J Med Educ. 1985;60:562–564.

Glass

. Undergraduate training in substance abuse in the United Kingdom. Br J Addict. 1989;84:197–202.

National Institute on Alcohol Abuse and Alcoholism (NIAAA). Resource Guide to the Evaluation of the Faculty Development Program in Alcohol and Other Drug Abuse. Part 1: Overview of the Evaluation Model. Rockville, MD: NIAAA; 1990.

International Centre for Drug Policy. Substance misuse in the undergraduate medical curriculum. April 2007. Available at: www.sgul.ac.uk/research/projects/cdp/our-work-programmes/pdfs/substance-misuse-book.pdf.

Massey

, Norris

, Lausin

, Nwaneri

, Lieberman

. Identifying harmful drinking using a single screening question in a psychiatric consultation-liaison population. Psychosomatics. 2011;52:362–366.

Gaughin

, Dodding

, White

, Ryan

. Changes in alcohol history taking and management of alcohol dependence by interns at The Royal Adelaide Hospital. Med Educ. 2000;34:170–174.

10.

Cape

, Hannah

, Sellman

. A longitudinal evaluation of medical student knowledge, skills and attitudes to alcohol and drugs. Addiction. 2006;101:841–849.

11.

Roche

, Parle

, Saunders

. Managing alcohol and drug problems in general practice: a survey of trainees’ knowledge, attitudes and educational requirements. Aust N Z J Public Health. 1996;20:401–408.

12.

Paul

, Yi

, John

, . Behavior, knowledge and attitudes toward Khat among Yemeni medical students and effects of a seminar. Subst Abus. 2012;33:373–377.

13.

Stack

, Lisa

, Arcand

, Briscoe

. Use of alcoholics anonymous as part of medical school education: students’ and educators’ perspectives. Subst Abus. 2012;33:387–391.

14.

Kahan

, Midmer

, Wilson

, Borsoi

. Medical students’ knowledge about alcohol and drug problems: results of the Medical Council of Canada examination. Subst Abus. 2006;27:1–6.

15.

Engs

RC.

The Student Alcohol Questionnaire. Bloomington, IN: Department of Health and Safety Education, Indiana University; 1975.

16.

Neumark

. Alcohol consumption in Israel: a public health and medical problem . [editorial]. Isr Med Assoc J. 2012;14:315–317.

17.

Engs

, Hanson

. The Student Alcohol Questionnaire: an updated reliability of the drinking patterns, problems, knowledge and attitude subscales. Psychol Rep. 1994;74:12–14.

18.

Sadock

, Sadock

. Alcohol related disorders. Synopsis of Psychiatry. 10th ed. Philadelphia, PA: Wolters Kluwer/Lippincott Williams & Wilkins; 2007:390–407.

19.

Ewing

. Detecting alcoholism: the CAGE questionnaire. JAMA. 1984;252:1905–1907.

20.

Cronbach

. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334.

21.

Sanson-Fisher

, Redman

, Walsh

, Mitchell

, Reid

, Perkins

. Training medical practitioners in information transfer skills: the new challenge. Med Educ. 1991;25:322–333.

22.

Bigby

, Barnes

. Evaluation of a faculty development program in substance abuse education. J Gen Intern Med. 1993;8:301–305.

23.

Chappel

, Jordan , Treadway

, Miller

. Substance abuse attitude changes in medical students. Am J Psychiatry. 1977;134:379–384.

24.

Ramsden

Learning to Teach in Higher Education. London: Routledge; 1992.

25.

Prosser

Trigwell

. Understanding Learning and Teaching: The Experience in Higher Education. Buckingham, UK: Society for Research into Higher Education and Open University Press; 1999.

26.

Anderson

, Clement . The AAPPQ revisited: the measurement of general practitioners’ attitudes to alcohol problems. Br J Addict. 1987;82:753–759.

27.

Geller

, Levine

, Mamon

Moore

, Bone

, Stokes

. Knowledge, attitudes and reported practices of medical students and house staff regarding the diagnosis and treatment of alcoholism. JAMA. 1989;261:3115–3120.

28.

D'Onofrio

, Nadel

, Degutis

, . Improving emergency medicine residents’ approach to patients with alcohol problems: a controlled educational trial. Ann Emerg Med. 2002;40:50–62.

29.

Silins

, Conigrave

, Ravkin

Dobbins

, Curry

. The influence of structured education and clinical experience on the attitudes of medical students toward substance misusers. Drug Alcohol Rev. 2007;26:191–200.

30.

Royal College of Psychiatrists. Alcohol and other drugs: core medical competencies. Final report of the Working Group of the Royal Colleges. June 2012. Available at: www.rpsych.ac.uk/publications/collegereports/op/op85.aspx.

31.

Morris-Wiliams

, Monrouxe

, Grant

, Edwards

. Teaching postgraduates about managing drug and alcohol misuse. BMJ. 2012;345:e5816. doi: 10;1136/bmj.e5816.