Differential Impact of a Crew Resource Management Program According to Professional Specialty

Abstract

Adverse events occur in 3% to 16% of hospital patients, half of these during surgery and related to human error. The authors’ objective was to determine the impact of a crew resource management program in collaboration with Swiss International Airlines. Participants included operating room personnel: surgeons, anesthesiologists, nurses, and technicians. Outcome measures were a 10-item questionnaire evaluating participants’ satisfaction and a 32-item survey to analyze participants’ learning. Nine seminars included 99 participants: 22% surgeons, 19% anesthesiologists, 29% nurses, and 30% nurse’s aides/technicians. Satisfaction was very high for course organization (91%), group dynamics (74%), and teaching methods (68%). Significant improvements in learning were observed after the course in 17 out of 32 questions. Surgeons demonstrated the greatest improvement in knowledge (P = .018), specifically teamwork and safety-related issues. Less improvement was seen for all specialties in stress recognition areas. Crew resource management is valuable in improving operating room staff knowledge regarding teamwork, safety climate, and stress recognition. However, program impact varies with participant specialty.

Keywords

crew resource management differential impact teamwork surgeons specialty

Three percent to 16% of patients will experience an adverse event during their hospital stay.^1-4 Between 28% and 51% of these events could be considered preventable and related to human error.^1,2,4,5 Human factors leading to errors have been studied extensively in aviation, and it has been known since the early 1980s that human weaknesses such as poor teamwork among cockpit crews contribute to half of all accidents.⁶ This led to the introduction of crew resource management (CRM) programs aimed at improving teamwork and communication among flight crews.⁷ These programs place a strong emphasis on the process of training crews to communicate and work together as a team.^8-10 Such programs aim to develop shared behaviors to improve safety, specifically team resources rather than individual resources. To reduce human error and improve teamwork and communication, an increasing number of hospitals have introduced CRM programs derived from aviation. The implementation of such programs has been associated with several organizational changes,¹¹ including enhanced personal commitment to patient safety, an increase in the use of checklists and self-reported incidents,¹² improved communication between surgeons and anesthesiologists,¹³ better operating room (OR) teamwork,¹⁴ fewer equipment issues, decreased OR delays,¹⁵ and, perhaps the most important benefit, reduced surgical mortality.¹⁶

CRM programs generally are organized as 1- or 2-day group sessions for all OR personnel (surgeons, anesthesiologists, nurses, technicians) to improve communication and coordination in multidisciplinary settings. However, they frequently are designed as stand-alone programs regardless of the participants’ professional culture, which can vary significantly depending on participants’ specialties and whether they are physicians, nurses, or technicians.¹⁷ It is currently unknown how these programs are perceived and how they affect participants, depending on their background and professional culture.

The purpose of this study was to analyze the impact of a CRM program on satisfaction and learning among OR personnel of a tertiary hospital and to compare its specific impact on surgeons, anesthesiologists, nurses, nurse’s aides, and technicians.

Methods

This study was performed in the Department of Surgery of the Geneva University Hospitals, Geneva, Switzerland. This is the only public hospital in the region. It is a 2200-bed health care center that provides primary and tertiary care to the canton of Geneva and the surrounding area (population of approximately 800 000). There were 12 979 surgical interventions performed in 2009. To improve surgical outcomes, we successively introduced a surgical safety checklist¹⁸ and a CRM program. The checklist was launched in July 2009 and comprised 28 items assessed at 3 points in time during the operative procedure (sign in, time out, sign out). The CRM program was introduced in November 2009 and was developed in collaboration with Swiss International Airlines. It was designed as a 1-day group seminar, chaired by an instructor from Swiss and a senior orthopedic surgeon with experience in medical education. Participation was compulsory for all OR staff members of the hospital (surgeons, anesthesiologists, nurses, nurse’s aides, and technicians). Each specialty was instructed to provide 2 to 3 persons in order to achieve a total of 10 to 12 persons for the seminar.

The course seminar is based on 3 interactive workshops complemented by specific lectures. The training day starts with an introductory course on safety in aviation, exploring similarities between aviation and surgery, and providing an overview of the mechanisms underlying medical errors. Participants are then invited to share personal errors as a starting point to an interactive workshop on error management and harm reduction. This course section is completed by a brief lecture on oral communication with special emphasis on the advantages of closed-loop communication and feedback. The second part of the training day centers on conflict resolution. Participants are involved in role-play to experience how the “win-lose” approach may endanger the ability to achieve a common goal, as compared to the “win-win” attitude. The last part of the workshop aims to improve participants’ understanding of the use and benefits of the surgical safety checklist. The day closes with an overview of the group dynamic and the phases of group formation.

To assess the impact of the training program on participants’ satisfaction and learning characteristics according to profession and specialty, 2 different questionnaires were administered to all participants of the course between November 2009 and September 2010. The first is a standardized 10-item questionnaire developed by the hospital training center. It explores 4 different areas of participants’ satisfaction—course organization, course content, group dynamic, and methods; it uses a 4-point Likert-type scale (dissatisfied to very satisfied). All participants were asked to complete the questionnaire at the end of the seminar.

The second questionnaire is aimed at assessing participants’ learning. It is based on a validated 32-item questionnaire¹⁴ that explores different aspects of CRM course content, particularly teamwork building, shared decision making in emergency situations, and other methods of improving patient safety. The answers are measured using a 5-point Likert-type scale (strongly disagree to strongly agree). The questionnaire was administered to participants at the beginning and at the end of the course to specifically measure the impact of the program on 1 or more dimensions. This questionnaire also included items on professional activity and individual participant characteristics.

Statistical Analysis

For descriptive analyses of participants’ characteristics and responses to the items on the satisfaction questionnaire we used percentages. Results were presented overall and separately for each profession. To assess knowledge improvement measured by the learning assessment questionnaire administered before and after the course, we calculated for each question mean preintervention and postintervention scores, the mean score difference (learning gain), and 95% confidence intervals (CI) as well as the corresponding effect sizes. This is defined as the mean unadjusted difference divided by the pooled standard deviation of the corresponding mean scores. Effect sizes of 0.2, 0.5, and 0.8 were regarded as small, medium, and large degrees of differences, respectively.¹⁹ A paired t test was used to compare the answers before and after course participation. In addition, mean score differences and their 95% CIs were calculated overall (sum of all mean score differences) and separately for the domains of teamwork, stress recognition, and safety culture, and the results were stratified by profession (anesthesiologists vs surgeons vs nurses vs nurse’s aides and technicians). Differences in learning gain between surgeons and the other professions were evaluated with use of the Student t test. We performed all analyses using the SPSS, version 15.0 (IBM, Chicago, IL).

Results

Between November 2009 and December 2010, a total of 99 participants attended the 9 seminars. The mean age was 41 years, and 53% of the participants were male (Figure 1).

Figure 1.

Characteristics of participants

The satisfaction questionnaire was completed by all participants (Figure 2), and the results were as follows: 95% of the participants valued their satisfaction with the overall course organization as high, 94.3% were satisfied to very satisfied with course content, 97.6% with group dynamics, and 95.3% with teaching methods. Nurses were the most satisfied among all professional groups for all measures except group dynamics, which was enjoyed the most by surgeons (Figure 3).

Figure 2.

Participants’ satisfaction with the crew resource management program

Figure 3.

Proportion of very satisfied participants, according to profession

The 32-item before/after course questionnaire analyzing participants’ learning was available for 70 (70.7%) of the participants. The results showed a very good level of learning among participants, with a significant change in 17 questions out of 32 and a change in effect size between 0.2 and 0.5 (Table 1). Knowledge improvement was observed for all professions with the greatest gain in learning in the domain of teamwork (Figure 4). Surgeons, when compared with the other professions, had the most substantial learning gain following program implementation (mean score difference 7.9 vs 3.5, P = .018), and more specifically in the domains of teamwork (mean score difference 5.0 vs 2.6, P = .025) and safety culture (mean score difference 2.1 vs 0.4, P = .026). Less gain was seen for all specialties in areas of stress recognition.

Table 1.

Learning Assessment Before and After the CRM Course

	Items	Mean Score, Pre (±SD)	Mean Score, Post (±SD)	Mean Score Difference (95% CI)	Effect Size	P Value
Q1	It is possible to perceive when colleagues start to lose control of the situation	3.86 (±0.822)	4.21 (±0.587)	0.36 (0.15; 0.56)	0.5	.001
Q2	Predicting what colleagues will do improves patient safety	4.16 (±0.987)	4.44 (±0.735)	0.29 (0.05; 0.53)	0.33	.021
Q3	To ensure patient safety, a shared knowledge of workload and available human resources is necessary	4.54 (±0.652)	4.80 (±0.403)	0.26 (0.08; 0.43)	0.49	.005
Q4	Human error cannot be avoided; it is important to learn how to deal with the consequences of human errors	3.79 (±1.329)	4.04 (±1.429)	0.26 (−0.06; 0.58)	0.19	.112
Q5	Not being aware of other professionals’ availabilities can negatively affect patient safety	4.67 (±0.583)	4.80 (±0.437)	0.13 (0.01; 0.24)	0.25	.028
Q6	A good professional does not make mistakes	1.57 (±0.941)	1.37 (±0.705)	−0.2 (−0.42; 0.02)	0.24	.07
Q7	Most accidents during patient care are due to a lack of professional skills	2.46 (±1.247)	2.43 (±1.258)	−0.03 (−0.30; 2.5)	0.02	.837
Q8	Rephrasing other’s sentences is a good way to ensure proper understanding	4.69 (±0.578)	4.74 (±0.440)	0.06 (−0.08; 0.19)	0.1	.398
Q9	Asking for help is a sign of lack of skills	1.14 (±0.460)	1.24 (±0.788)	0.1 (−0.06; 0.26)	0.16	.211
Q10	To ensure patient safety I must make sure others have properly understood my wording	4.53 (±0.675)	4.73 (±0.479)	0.2 (0.03; 0.37)	0.35	.022
Q11	When in doubt, asking a colleague is proof of ineptitude	3.77 (±1.553)	3.64 (±1.711)	−0.13 (−0.49; 0.23)	0.08	.474
Q12	To improve patient safety, I need to assess others’ availabilities	4.47 (±0.557)	4.59 (±0.577)	0.11 (−0.03; 0.26)	0.21	.117
Q13	Conflict between health care professionals is deleterious to patients	4.69 (±0.498)	4.71 (±0.617)	0.03 (−0.14; 0.20)	0.04	.741
Q14	In emergency situations, only the patients’ condition matters	3.26 (±1.380)	3.43 (±1.325)	0.17 (−0.12; 0.46)	0.13	.237
Q15	Understanding other professionals’ tasks is a good way to improve mutual collaboration	4.76 (±0.432)	4.87 (±0.337)	0.11 (0.01; 0.22)	0.29	.031
Q16	To ensure patient safety, I need to take into account other professionals’ needs	4.53 (±0.653)	4.69 (±0.468)	0.16 (0.03; 0.28)	0.29	.015
Q17	To improve one’s knowledge it is necessary to be able to talk about one’s mistakes	4.74 (±0.440)	4.84 (±0.367)	0.1 (−0.01; 0.21)	0.25	.07
Q18	Dysfunctioning teams are a major cause of accidents	4.00 (±0.816)	4.09 (±0.676)	0.09 (−0.11; 0.28)	0.12	.39
Q19	To improve patient care, each health care professional should be able to express one’s own perspective	4.40 (±0.646)	4.53 (±0.583)	0.13 (−0.03; 0.29)	0.21	.106
Q20	To develop a shared understanding of the different degrees of an emergency situation makes team coordination easier	4.44 (±0.694)	4.70 (±0.492)	0.26 (0.11; 0.41)	0.44	.001
Q21	Patient safety is not affected by interruptions	2.41 (±1.136)	2.33 (±1.086)	−0.09 (−0.37; 0.20)	0.07	.544
Q22	Patient safety is before all a team matter	4.60 (±0.668)	4.77 (±0.423)	0.17 (0.01; 0.33)	0.31	.039
Q23	Shared anticipation to deal with complex situations improves work processes	4.54 (±0.606)	4.70 (±0.521)	0.16 (0.03; 0.28)	0.28	.015
Q24	A competent leader is able to explain his decisions	4.37 (±0.705)	4.66 (±0.587)	0.29 (0.16; 0.41)	0.44	0
Q25	Accidents often results from successive mishaps	4.44 (±0.715)	4.57 (±0.693)	0.13 (0.001; 0.26)	0.19	.049
Q26	A good professional is not sensitive to stress	1.80 (±0.878)	1.76 (±0.806)	−0.04 (−0.25; 0.16)	0.05	.678
Q27	When cases are complex, professional teams have greater difficulty dealing with other patients	3.56 (±1.030)	3.63 (±1.024)	0.07 (−0.15; 0.29)	0.07	.512
Q28	Fatigue does not impair professional skills	1.43 (±0.714)	1.44 (±0.773)	0.01 (−0.16; 0.19)	0.01	.871
Q29	Multidisciplinary meetings help improve patient care	4.46 (±0.582)	4.64 (±0.483)	0.19 (0.05; 0.32)	0.34	.008
Q30	Human communication is always ambiguous	3.16 (±1.187)	3.43 (±1.314)	0.27 (−0.01; 0.55)	0.22	.058
Q31	In emergency situations it is acceptable to help each other beyond professional boundaries	4.20 (±0.972)	4.57 (±0.753)	0.37 (0.20; 0.54)	0.43	0
Q32	Teams learn from their mistakes	4.77 (±0.423)	4.93 (±0.259)	0.16 (0.06; 0.25)	0.45	.002

Abbreviations: CRM, crew resource management; SD, standard deviation; CI, confidence interval.

Figure 4.

Learning gain after the crew resource management program

Discussion

We introduced a multidisciplinary CRM program to improve teamwork and coordination in the OR of the Geneva University Hospitals. Most of the participants valued this experience highly and rated their level of satisfaction as high or very high for course organization, course content, group dynamics, and knowledge acquisition. Participants also improved their knowledge concerning teamwork and patient safety-related issues, with surgeons demonstrating the greatest learning gain following program participation when compared with other specialties.

These results are similar to previous studies showing that surgical attendings and residents are usually highly satisfied with the level of collaboration developed with other specialties and professions working in the OR, particularly with nurses.^20-24 A study by Sexton et al²³ revealed that surgeons had the highest level of positive perception of teamwork, communication, and collaboration in the OR compared with other specialties and professions. Approximately 64% of surgical attendings and 73% of surgical residents rated the rate of communication, collaboration, and cooperation with other surgeons as high, when compared with 39% of anesthesiologists, 28% of surgical nurses, and 25% of anesthesia nurses. This may be explained by discrepancies in the perception of what represents good teamwork and collaboration among nurses, anesthesiologists, and surgeons. Surgeons usually are more supportive of a steep hierarchy between junior and senior staff than those in other specialties.²² This is not what was taught in our program as it was designed to promote flat hierarchy and horizontal interprofessional collaboration. As a result, surgeons may have modified their understanding of good-quality teamwork and thus improved their knowledge in this area to a greater degree than other specialties. Alternatively, these differences may be explained by a disconnect between surgeons’ beliefs about their personal teamwork and effectiveness skills and the perceptions of these qualities by the rest of the team. Thus, surgeons probably learned more from the program than nurses or anesthesiologists, who already considered the definition of good collaboration to be when everyone’s input is respected.²⁵

Another interesting finding of our study is that the overall learning gain was higher for surgeons than for anesthesiologists or nurses. This discrepancy raises interesting questions about the reasons for such differences. It may be related to the fact that this kind of training was offered to surgeons for the first time and that their baseline level of knowledge about the course topic was lower. Because anesthesiologists already had such training during their careers, their baseline level of knowledge was higher and the learning gain was less important. This hypothesis is also compatible with the analysis of the satisfaction questionnaire, which showed that twice as many anesthesiologists as surgeons (20% vs 10%, respectively) had no opinion about the learning benefits following the CRM program.

For nurse’s aides and technicians, the results are more difficult to explain. They had the lowest level of satisfaction following the program, and their levels of learning in safety and stress recognition areas were particularly low. This may be related to the fact that most workshops organized during the CRM program were oriented mainly toward patient outcomes and decision-making processes. These are 2 areas in which their interest may have been limited. Furthermore, several documents used in the program were available only in English, a language with which the majority of nurse assistants and technicians are not familiar, and this may have presented a barrier to learning.

Our study has several limitations. First, it was undertaken in a single hospital and analyzed a specific CRM-based training program. Its generalization to other CRM-based programs in other multidisciplinary settings may be limited. Furthermore, even if course participation was compulsory for all OR staff members of the hospital, we cannot exclude a possible selection bias of participants more inclined to participate, enjoy, and learn from such a training program. In addition, the results of this study reflect participants’ perceptions of the course and their newly acquired knowledge on the impact of organization in the OR, but do not evaluate the impact of such learning in the OR itself. Finally, the groups were small; however, small sample sizes are common for team research in operational settings, with the number of teams evaluated typically ranging from 5 to 15.²⁶ In addition, they represent a substantial proportion of all employees in the Department of Surgery, because nearly 20% of physicians and 50% of nurses have taken the course.

Despite these limitations, the results indicate that professionals highly valued the CRM-based approach and significantly improved their knowledge of teamwork and shared decision making. In the long term, this may improve the overall organizational safety climate and patient safety. CRM-based training programs are highly recommended for routine and emergency practice in multidisciplinary areas where a high level of coordination and collaboration is required. However, when designing these programs it should be understood that their impact may vary depending on the participants’ professions and preliminary knowledge of the different areas discussed in the course. This aspect must be considered when developing such a program for a multidisciplinary audience; these programs should be adapted to the professional background of the participants. Future research should focus on the impact of specifically designed CRM-based training programs aimed at improving interprofessional teamwork and communication in multidisciplinary environments.

Footnotes

Acknowledgements

The authors would like to thank Frédéric Macheret, François Emery, and Manuel Schmidt from Swiss Airlines; Richard Stern, MD, for editorial advice; and Beatrice Moesching for organizing the seminars.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Brennan

Leape

Laird

. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324:370-376.

Wilson

Runciman

Gibberd

Harrison

Newby

Hamilton

. The Quality in Australian Health Care Study. Med J Aust. 1995;163:458-471.

Baker

Norton

Flintoft

. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ. 2004;170: 1678-1686.

Vincent

Neale

Woloshynowych

. Adverse events in British hospitals: preliminary retrospective record review. BMJ. 2001;322:517-519.

Leape

Brennan

Laird

. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med. 1991;324:377-384.

US General Accounting Office. Human Factors: FAA’s Guidance and Oversight of Pilot Crew Resource Management Training Can Be Improved (Report No. GAO/RCED-98-7). Washington, DC: US General Accounting Office; 1997.

Helmreich

Merritt

Wilhelm

. The evolution of crew resource management training in commercial aviation. Int J Aviat Psychol. 1999;9:19-32.

Cooper

White

Lauber

. Resource Management on the Flightdeck: Proceedings of a NASA/Industry Workshop. Moffet Field, CA: NASA, Ames Research Center; 1980:102.

Wiener

Kanki

Helmreich

. Cockpit Resource Management. San Diego, CA: Academic Press; 1993.

10.

Helmreich

. On error management: lessons from aviation. BMJ. 2000;320:781-785.

11.

Zeltser

Nash

. Approaching the evidence basis for aviation-derived teamwork training in medicine. Am J Med Qual. 2010;25:13-23.

12.

Sax

Browne

Mayewski

. Can aviation-based team training elicit sustainable behavioral change? Arch Surg. 2009;144:1133-1137.

13.

Awad

Fagan

Bellows

. Bridging the communication gap in the operating room with medical team training. Am J Surg. 2005;190:770-774.

14.

Haller

Garnerin

Morales

. Effect of crew resource management training in a multidisciplinary obstetrical setting. Int J Qual Health Care. 2008;20:254-263.

15.

Wolf

Way

Stewart

. The efficacy of medical team training: improved team performance and decreased operating room delays: a detailed analysis of 4863 cases. Ann Surg. 2010;252:477-483.

16.

Neily

Mills

Young-Xu

. Association between implementation of a medical team training program and surgical mortality. JAMA. 2010;304:1693-1700.

17.

Hall

. Interprofessional teamwork: professional cultures as barriers. J Interprof Care. 2005;19(suppl 1):188-196.

18.

Haynes

Weiser

Berry

. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360:491-499.

19.

Cohen

. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Earlbaum; 1988.

20.

Carney

Mills

Bagian

Weeks

. Sex differences in operating room care giver perceptions of patient safety: a pilot study from the Veterans Health Administration Medical Team Training Program. Qual Saf Health Care. 2010;19:128-131.

21.

Carney

West

Neily

Mills

Bagian

. Differences in nurse and surgeon perceptions of teamwork: implications for use of a briefing checklist in the OR. AORN J. 2010;91:722-729.

22.

Makary

Sexton

Freischlag

. Operating room teamwork among physicians and nurses: teamwork in the eye of the beholder. J Am Coll Surg. 2006;202: 746-752.

23.

Sexton

Thomas

Helmreich

. Error, stress, and teamwork in medicine and aviation: cross sectional surveys. BMJ. 2000;320:745-749.

24.

Thomas

Sexton

Helmreich

. Discrepant attitudes about teamwork among critical care nurses and physicians. Crit Care Med. 2003;31:956-959.

25.

Sexton

. The Better the Team, the Safer the World: Golden Rules of Group Interaction in High-Risk Environments. Ladenburg, Germany: Swiss Re Centre for Global Dialog, and the Gottlieb Daimler and Karl Benz Foundation; 2004.

26.

Morey

Simon

Jay

. Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTeams project. Health Serv Res. 2002;37: 1553-1581.