Meta-Analysis of Surgical Safety Checklist Effects on Teamwork,Communication,Morbidity,Mortality,and Safety

Abstract

The purpose of this study is to examine the effectiveness of surgical safety checklists on teamwork, communication, morbidity, mortality, and compliance with safety measures through meta-analysis. Four meta-analyses were conducted on 19 studies that met the inclusion criteria. The effect size of checklists on teamwork and communication was 1.180 (p = .003), on morbidity and mortality was 0.123 (p = .003) and 0.088 (p = .001), respectively, and on compliance with safety measures was 0.268 (p < .001). The results indicate that surgical safety checklists improve teamwork and communication, reduce morbidity and mortality, and improve compliance with safety measures. This meta-analysis is limited in its generalizability based on the limited number of studies and the inclusion of only published research. Future research is needed to examine possible moderating variables for the effects of surgical safety checklists.

Keywords

checklist surgery surgical safety teamwork communication morbidity mortality safety

In the United States, 51.4 million ambulatory surgery visits and 48 million inpatient surgeries are performed annually (Centers for Disease Control and Prevention [CDC], 2011; Cullen, Hall, & Golosinskiy, 2009). This results in the average person undergoing 3.4 inpatient operations and 2.6 outpatient operations in his or her lifetime (Lee, Regenbogen, & Gawande, 2008). The operating room is one of the most common locations for medical errors and adverse events with one half to two thirds of all errors being attributed to surgical care (Gawande, Zinner, Studdert, & Brennan, 2003; Leape et al., 1991; Rogers et al., 2006). With over 275,000 surgical procedures performed daily, an emphasis on perioperative safety is a necessity. The most commonly cited cause of surgical error is breakdown in communication (Lingard et al., 2004; Makary et al., 2006). One documented method to improve communication and reduce errors and adverse events is the surgical safety checklist, introduced by the World Health Organization (WHO) (Haynes et al., 2009).

Checklists are designed to improve patient outcomes by providing a visual tool for standardized communication (Haynes et al., 2009). The WHO conducted the first study of surgical safety checklists and found that the use of the checklist reduced morbidity and mortality by over 30% (Haynes et al., 2009). Since the release of this landmark study, hospitals have adapted and implemented checklists to improve patient outcomes. Moreover, the Centers for Medicare & Medicaid Services (CMS) recommended the use of surgical safety checklists as a quality measure in 2016.

Despite the increased use of surgical safety checklists, research regarding the effects of surgical safety checklists has been varied. Researchers have examined the effects of surgical safety checklists on antibiotic prophylaxis administration (Askarian, Kouchak, & Palenik, 2011; de Vries, Dijkstra, Smorenburg, Meijer, & Boermeester, 2010; Paull et al., 2010), perception of teamwork (Böhmer et al., 2012; Haynes et al., 2011; Helmiö et al., 2011; Kearns et al., 2011; Makary et al., 2007; Paige, Aaron, Yang, Howell, & Chauvin, 2009; Wolf, Way, & Stewart, 2010), postoperative complications (Askarian et al., 2011; de Vries, Prins, et al., 2010; Haynes et al., 2009; Young-Xu et al., 2011), in-hospital mortality (de Vries,Prins, et al., 2010; Haynes et al., 2009), employee satisfaction (Böhmer et al., 2012), wrong site surgery (Makary et al., 2007), communication (Awad et al., 2005; Helmiö et al., 2011; Kearns et al., 2011; Makary et al., 2007), and operating room delays (Wolf et al., 2010). Most studies identified significant improvements in the surgical process and patient outcomes but the size of the improvement varied greatly across the studies. No comprehensive review or meta-analysis has been conducted to provide a measurement of the effect of these checklists on teamwork, communication, patient outcomes, and compliance with recommended safety measures. To overcome this gap in the state of the science about surgical safety checklists, the researchers conducted a meta-analysis that quantitatively synthesizes the current literature. This article reports the meta-analysis results on the effect of surgical safety checklists on teamwork, communication, morbidity, mortality, and compliance with operating room safety measures.

Method

Meta-analysis is a method to quantitatively combine research findings from multiple studies that allow researchers to measure the effects of interventions on outcomes from multiple studies thus increasing the power and precision of study findings (Cooper, Hedges, & Valentine, 2009). The following research questions guided this meta-analysis

Research Question 1: What is the effect of surgical safety checklists on teamwork and communication?

Research Question 2: What is the effect of surgical safety checklists on patient morbidity and mortality?

Research Question 3: What is the effect of surgical safety checklists on compliance with recommended surgical safety measures?

Sample Inclusion and Exclusion Criteria

In meta-analysis, it is essential to use diverse search strategies to avoid introduction of bias from narrow searches (Cooper et al., 2009). For this meta-analysis, the researcher conducted a database search of CINAHL, ProQuest, and MEDLINE databases using the search terms “checklist,” “operating room,” “surgery,” “perioperative,” “surgical,” “teamwork,” and “communication tools.” This search included an author search for the Checklist Manifesto author Atul Gawande (Gawande, 2010) and Alex Haynes, the principal investigator of the WHO study (Haynes et. al, 2009). The researcher also conducted a search for studies that were previously cited in the research studies. Finally, a Google Scholar search identified any additional research studies, conference, and presentations that may have been missed using other search strategies.

Research studies were included if they were in English, conducted in inpatient or ambulatory operating room settings, used a surgical safety checklist conducted immediately prior to incision by the operating team, used communication and teamwork measures that had documented reliability and validity, or reported complications related to surgery as identified by the American College of Surgeons’ National Surgical Quality Improvement Program (Khuri et al., 1995). The complications included from the aforementioned program are acute renal failure, bleeding requiring the transfusion of four or more units of red cells within the first 72 hr after surgery, cardiac arrest requiring cardiopulmonary resuscitation, coma of 24 hr duration or more, deep-vein thrombosis, myocardial infarction, unplanned intubation, ventilator use for 48 hr or more, pneumonia, pulmonary embolism, stroke, major disruption of wound, infection of surgical site, sepsis, septic shock, the systemic inflammatory response syndrome, unplanned return to the operating room, vascular graft failure, or death. Mortality was defined as patient death in or out of the hospital from any cause within 30 days after surgery (Grover et al., 1994). Recommended surgical safety measures were defined as the 19 evidence-based practices indicated on the WHO’s Surgical Safety Checklist as outlined in Table 1 (World Health Organization [WHO], 2008). Finally, studies that utilized a two-group postintervention comparison or a one-group pretest–posttest design and reported sufficient statistical results to be able to calculate an effect size were included.

Table 1.

WHO Surgical Safety Checklist.

Elements of the WHO Surgical Safety Checklist
Before induction of anesthesia The patient verifies his or her identity, surgical site, and procedure and the consent is completed The surgical site is marked, if applicable The pulse oximeter is on the patient and functioning Any allergies are communicated to all team members The patient’s airway and risk of aspiration is addressed and all applicable supplies and assistance are available If there is a risk of at least 500 ml blood loss, IV access and fluids are available
Before skin incision All team members are introduced by name and role The patient’s identity, surgical site, and procedure are confirmed Anticipated critical events are reviewed Surgeon reviews critical and unexpected steps, operative duration, and anticipated blood loss Anesthesia provider reviews concerns Nursing staff reviews confirmation of sterility, equipment availability, and other concerns Prophylactic antibiotics are administered no longer than 60 min before incision if indicated All essential imaging is available
Before the patient leaves the operating room Nursing staff reviews the following: Name of procedure is recorded correctly All needle, sponge, and instrument counts are completed and corrected, if applicable Any specimens are labeled correctly, including the patient’s name Any equipment issues are addressed The surgeon, nurse, and anesthesia provider reviews any key concerns for the recovery and care of the patient

Source. Adapted from Haynes et al. (2009).

Note. IV = intravenous.

Data Management Procedures

As research studies were located, abstracts were examined for inclusion in the meta-analysis. Eligibility for the meta-analysis was based on the previously identified inclusion and exclusion criteria. EndNote, a bibliographic management software program, was used to organize publications. Full text documents were collected and saved as PDF documents attached to the reference in EndNote. Using the abstract, studies were categorized into one of three categories: not eligible, possibly eligible, and eligible. All search strategies and results were saved and documented for record keeping. Full text documents of abstracts categorized as possibly eligible were retrieved, read, and judged as eligible or not eligible. A hard copy of each research study was printed out and organized in a file cabinet drawer dedicated to the meta-analysis project. The final studies in the eligible category were coded in the codebook for inclusion in the meta-analysis.

Data Coding and Analysis

Coded variables included the following: (a) characteristics of the study setting (teaching status, number of operating rooms, number of licensed beds), (b) surgical checklist details (components, training, time of use, type of surgery), and (c) outcomes (time until final measurement, teamwork measures, communication measures, morbidity measures). In addition, data needed to complete effect size estimations (sample size, means, standard deviation, standard error, direction of effect) were also coded for each study. Each study was coded by one researcher on two separate occasions. The coding was then compared and no differences were found.

The standardized mean difference of postintervention scores (d) was used to estimate effect sizes (Cooper, 2010). Conceptually, a standardized mean difference is the mean of the treatment group minus the mean of the control group, divided by the pooled standard deviation. The effect size is a unitless measure which can be averaged across primary studies using different measures of constructs. A random-effects model was used for data analysis because of the heterogeneity across studies to include both within-study and between-study variance (Cooper et al., 2009). The random-effects model assumes that the true effect size varies across different studies based on different confounding variables (Cooper et al., 2009). Heterogeneity was expected because of site, intervention, and measurement variables. Statistical heterogeneity was calculated using the Q and I² statistics for between-study variation and observed variation due to real differences in effect size. Calculations were completed using an Excel spreadsheet designed with the required formulas for statistical analysis. A significance level of p < .05 was used for this study.

Results

Figure 1 illustrates the flow diagram of the literature search. A total of 55 articles were identified from the search. There were 36 articles excluded because data needed for the meta-analysis were not included. These missing data included statistical results needed for effect size calculations (28 studies), eligible outcome variables (5 studies), or not using a checklist as the intervention (3 studies). The remaining 19 studies were fully eligible for inclusion in the meta-analysis, and are listed in Table 2. The research sample varied based on the goals of the study. Some studies examined patients while others examined surgical procedures. These samples differed because patients may undergo multiple surgical procedures; thus, procedures may include a single patient multiple times. Finally, studies that examined teamwork and communication sampled employees or staff members.

Table 2.

Surgical Safety Checklist Studies Included in the Sample.

Author and Year of Publication	Sample Size	Sample Type	Outcome Measures	Focus of Analysis
Yuan et al., 2012	Control = 232 Treatment = 249	Patients	Compliance with 4 or more safety measures (airway assessment, use of pulse oximetry, IV placement, antibiotic prophylaxis, completion of surgical pause, completion of sponge count) Death within 30 days of surgery Occurrence of any major complication within 30 days of surgery	Compliance with safety measures
Askarian, Kouchak, & Palenik, 2011	Control = 144 Treatment = 150	Patients	Incidence of major postoperative complications within 30 days of surgery Compliance with 3 safety measures (sign in, time out, and sign out)	Compliance with safety measures
Haynes et al., 2011	Control = 281 Treatment = 257	Surgical team members	Surgical team members’ perceptions of teamwork (Safety Attitudes Questionnaire)	Teamwork and communication
Weiser et al., 2010	Control = 842 Treatment = 908	Patients	Major complications within 30 days of surgery Mortality within 30 days of surgery	Morbidity and mortality
de Vries, Prins, et al., 2010	Control = 3,760 Treatment = 3820	Patients	Complications occurring during hospitalization postoperative mortality during hospitalization	Morbidity and mortality
Böhmer et at., 2012	Control = 71 Treatment = 71	Surgical team members	Surgical team members’ perception of interprofessional cooperation based on an attitude survey	Teamwork and communication
Kearns et al., 2011	Control = 53 Treatment = 46	Surgical team members	Surgical team members’ perceptions of improved communication through a questionnaire	Teamwork and communication
de Vries, Dijkstra, Smorenburg, Meijer, & Boermeester, 2010	Control = 369 Treatment = 403	Surgical procedures	Antibiotic prophylaxis administered 30 min or more before incision	Compliance with safety measures
Calland et al., 2009	Control = 23 Treatment = 24	Surgical procedures	Equipment and personnel communication determined through observation	Teamwork and communication
Haynes et al., 2009	Control = 3,733 Treatment = 3955	Patients	Occurrence of any major complication, including death, during the period of postoperative hospitalization up to 30 days	Morbidity and mortality
Wolf, Way, & Stewart, 2010	Control = 72 Treatment = 44	Surgical team members	Surgical team member’s perceptions of teamwork climate though the Safety Attitudes Questionnaire	Teamwork and communication
Paull et al., 2010	Control = 74 Treatment = 74	Facilities	Compliance with prophylactic antibiotic administration within 60 min of incision	Compliance with safety measures
Makary et al., 2007	Control = 306 Treatment = 116	Surgical team members	Surgical team members’ perceptions of team coordination through the Safety Attitudes Questionnaire	Teamwork and communication
Young-Xu et al., 2011	Control = 42 Treatment = 42	Patients	Observed and risk-adjusted 30-day morbidity rates	Morbidity
Neily et al., 2010	Control = 74 Treatment = 74	Patients Facilities	Observed and risk-adjusted 30 day mortality rates	Mortality
Awad et al., 2005	Control = 213 Treatment = 213	Surgical team members	Perceptions of communication among anesthesia, surgeon, and nurse through a communication survey	Teamwork Communication
Helmiö et al., 2011	Control = 304 Treatment = 443	Surgical team members	Surgical team members’ perceptions of successful communication through a questionnaire	Teamwork and communication
Paige et al., 2008	Control = 20 Treatment = 20	Surgical team members	Peer assessment of team performance through the Operating Room Teamwork Assessment Scale	Teamwork and communication
Paige, Aaron, Yang, Howell, & Chauvin, 2009	Control = 36 Treatment = 36	Surgical team members	Surgical team members’ perceptions of teamwork through the Operating Room Teamwork Assessment Scale	Teamwork and communication

Note. IV = intravenous.

There were a number of differences in the studies. The sample included national and international studies with three studies including multiple countries. The majority of studies were from the United States (n = 9); the rest were from Great Britain (n = 1), the Netherlands (n = 2), Liberia (n = 1), Iran (n = 1), Germany (n = 1), and Finland (n = 1). Three studies conducted by the WHO had multiple countries in the sample (Canada, India, Jordan, New Zealand, the Philippines, Tanzania, England, and the United States). Ten studies used a single-group, pretest–posttest design (Awad et al., 2005; Böhmer et al., 2012; Haynes et al., 2011; Helmiö et al., 2011; Kearns et al., 2011; Makary et al., 2007; Paige et al., 2008, 2009; Paull et al., 2010; Wolf et al., 2010), five studies used a two-group, pretest–posttest design (Askarian et al., 2011; de Vries, Prins, et al., 2010; Haynes et al., 2009; Weiser et al., 2010; Yuan et al., 2012), three studies used a retrospective analysis design (de Vries, Dijkstra, et al., 2010; Neily et al., 2010; Young-Xu et al., 2011), and one study used a randomized control trial (Calland et al., 2011). Sample size varied from 20 to over 3,000, and the units of analysis were hospitals, surgical team members, patients, and surgical procedures. Some studies examined specific types of complications such as pneumonia, while others focused on all postoperative complications. Samples were calculated as a single unit regardless of the type of sampling unit used in the study with regard to statistical analysis. Teamwork and communication was measured with the most variety due to the availability of different tools to measure teamwork and communication. Heterogeneity was interpreted by I² and classified as low (25%), moderate (50%), or high (75%).

The results of the four meta-analyses on the outcome measures are displayed in Table 3. A positive effect size indicates an improvement in outcomes. For the effect of checklists on teamwork and communication, 10 studies were included in the analysis. The summary effect size of these studies was 1.180 (SE = 0.392, 95% confidence interval [CI] [0.411, 1.999], p = .003). Teamwork and communication also demonstrated high heterogeneity (Q = 638.500, p = .035, I² = 98.6%). Seven primary studies were included in the meta-analysis for morbidity. The summary effect size for morbidity was 0.123 (SE = 0.041, 95% CI [0.043, 0.204], p = .003). Morbidity analysis also showed a high level of heterogeneity (Q = 13.474, p = .035, I² = 77.7%). Mortality analysis consisted of four studies with a summary effect size of 0.088 (SE = 0.026, 95% CI [0.038, 0.139], p = .001) indicating reduced mortality. The results of the analysis of mortality identified a moderate amount of heterogeneity (Q = 6.013, p = .035, I² = 50.1%). Finally, the compliance meta-analysis for effect size included four studies. The summary effect size was 0.268 (SE = 0.052, 95% CI [0.166, 0.370], p < .001) and the analysis found low heterogeneity (Q = 3.193, p = .035, I² = 6%). Forest plots of data are available from the corresponding author.

Table 3.

Results of Surgical Safety Checklist Meta-Analyses.

Outcome Measure	No. of Included Studies (k)	Mean Effect Size (d)	p Value	SE	95% CI Lower Limit	95% CI Upper Limit	Q	p Value of Q	I ²
Teamwork and communication	10	1.151	.004	0.404	0.360	1.943	677.947	.035	98.7
Morbidity	4	−0.162	.003	0.044	−0.250	−0.075	26.020	.035	76.9
Mortality	4	−0.088	.001	0.026	−0.139	−0.038	6.013	.035	50.1
Compliance with safety measures	4	0.134	.220	0.127	−0.115	0.384	17.744	.035	83.1

Note. CI = confidence interval.

Discussion

This study sought to measure the effect of surgical safety checklists on teamwork and communication, morbidity, mortality, and compliance with safety measures. The positive effect found in this study indicated improvement in outcome variables following the checklist intervention. The meta-analysis identified that the use of a surgical safety checklist reduced morbidity and mortality, and improved teamwork, communication, and compliance with safety measures in operating rooms. These results should be interpreted with some caution because the heterogeneity found in the analyses indicates large variability in the studies which could inflate the effect size.

The strongest effect of checklists was on teamwork and communication. As previously noted, surgical error is commonly attributed to breakdown in communication (Lingard et al., 2004; Makary et al., 2006) and surgical safety checklists were originally designed to improve communication between surgical team members (Awad et al., 2005; Carney, West, Neily, Mills, & Bagian, 2010; Haynes et al., 2009). These results confirm that surgical safety checklists are effective in improving communication and potentially reducing surgical error and adverse events since a significant number of these are caused by a breakdown in communication (Gawande et al., 2003; Rogers et al., 2006; The Joint Commission, 2013).

Surgical mortality is estimated to be 21 out of every 1,000 surgical procedures (Fecho, Lunney, Boysen, Rock, & Norfleet, 2008). In addition, with almost 100 million surgical procedures performed annually in the United States, roughly 1,000,000 patients will die within 30 days of surgery (CDC, 2011). While the effect size for morbidity and mortality was small, given the number of surgical procedures performed each year, even a small improvement in these outcomes would save lives and reduce the burden of complications.

This meta-analysis also found a significant improvement in compliance with safety measures with the use of surgical safety checklists. As with communication, this result was expected because a checklist provides a visual reminder of recommended safety measures, reducing the reliance on memory and improving compliance (Abdel-Rehim, Morritt, & Perks, 2011; Rydenfält, Johansson, Odenrick, Åkerman, & Larsson, 2013). For example, a surgical safety checklist provides a reminder to surgical team members to ask the patient about surgical site, or administer a prophylactic antibiotic within 60 min of incision. These reminders increase compliance with safety recommendation resulting in reduced incidence of surgical site infections.

Teamwork, communication, morbidity, and mortality were significantly heterogeneous. This finding was not surprising and occurred because of the varied units of analysis of the studies, for example, when measuring morbidity, samples were surgical procedures, patients, or hospitals. Mortality had a moderate variability but was limited in sample type to hospitals or patients. This can explain why there is less variability for mortality. Patients can experience multiple complications from one procedure or patients may undergo multiple procedures. Mortality is a single event relating back to only one patient and one procedure. Since the number and types of outcomes vary more for morbidity than mortality, it is logical to see a larger variance in morbidity than mortality. In contrast, compliance with safety measures showed low heterogeneity. The samples in all of the included studies were adults undergoing surgical procedures. The consistency of the units of analysis can explain the low heterogeneity.

Future research on the effect of surgical safety checklists would be strengthened by the conduction of a moderator analysis. The publications range in dates from 2005 through 2012 showing the relative newness of checklists as an intervention, and more research is needed to complete a reliable moderator analysis. Possible moderators identified in this study include length of time between implementation and evaluation, size of facility, type of surgical procedure, content of the checklist, and type of facility. While possible moderators were identified during the conduction of this meta-analysis, a moderator analysis was not conducted due to the small number of available studies for inclusion. Conducting a moderator analysis would provide evidence about which characteristics maximize the benefits of the use of a checklist. As more primary studies accrue, adequate studies will be available to conduct moderator analysis.

There are limitations to this meta-analysis. First, only published research was included. There is a tendency in scholarly publications to publish research that results in significant findings (De Oliveira, Chang, Kendall, Fitzgerald, & McCarthy, 2012; Dwan et al., 2008). Studies that fail to find significant results are either not submitted or not published. This tendency could result in nonsignificant findings being excluded due to a possible publication bias. Next, there were 28 studies with statistically significant findings that were not included due to a lack of reported data necessary to calculate an effect size. The inclusion of these studies may have significantly increased the effect size of surgical safety checklists. Another limitation was the lack of a second coder to validate the coding of the studies. Finally, the studies used different sampling units: Nine studies used surgical team members, six studies used patients, three studies used surgical procedures, and two studies used facilities. This difference could have an impact on the effect as patients may undergo multiple surgical procedures or experience multiple complications making a comparison of patients with procedures, or complications difficult; however, the random-effects model compensates for these differences in units of analysis. The studies were screened to ensure that multiple results from the same study were not included in the analyses.

Limitations of the primary studies themselves could also affect the findings of this meta-analysis. An increase in awareness of communication resulting from training on the use of the checklist could have improved communication even without the use of the checklist thus increasing the measured effect of the checklist for communication. The measures used in primary studies frequently required the use of subjective opinions of team members and may have presented a bias into the subjective findings as participants may respond with socially acceptable answers or give responses that they feel the researcher is expecting. Compliance with safety measures could have been influenced by potential observer bias as practitioners may have changed their practice when being monitored or observed increasing the measured effect of surgical safety checklists. Finally, the studies included the use of surgical safety checklists that were designed and personalized by each facility to meet their needs. While the checklists included much of the same content (patient identity, procedure, preoperative antibiotic administration, venous thrombosis prevention, etc.), there may have been other characteristics in the checklist or how it was implemented that affected the findings and reduced the generalizability.

This is the first study to our knowledge that quantitatively synthesized the effect of surgical safety checklists on teamwork, communication, morbidity, mortality, and compliance with safety measures. Findings from this meta-analysis indicated that the use of surgical safety checklists resulted in significant positive effects on teamwork and communication, morbidity, and mortality but not on compliance with safety measures. Future research is needed to identify moderating variables that can positively influence the effect of surgical safety checklists on patient outcomes. The use of the surgical safety checklist as a safety measure is growing exponentially and more research about how to maximize the benefits of this tool is needed.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Abdel-Rehim

Morritt

Perks

(2011). WHO surgical checklist and its practical application in plastic surgery. Plastic Surgery International. Advance online publication. doi:10.1155/2011/579579

Askarian

Kouchak

Palenik

C. J.

(2011). Effect of surgical safety checklists on postoperative morbidity and mortality rates, Shiraz, Faghihy Hospital, a 1-year study. Quality Management in Health Care, 20, 293-297.

Awad

S. S.

Fagan

S. P.

Bellows

Albo

Green-Rashad

Garza

Berger

D. H.

(2005). Bridging the communication gap in the operating room with medical team training. American Journal of Surgery, 190, 770-774. doi:10.1016/j.amjsurg.2005.07.018

Böhmer

A. B.

Wappler

Tinschmann

Kindermann

Rixen

Bellendir

Gerbershagen

M. U.

(2012). The implementation of a perioperative checklist increases patients’ perioperative safety and staff satisfaction. Acta Anaesthesiologica Scandinavica, 56, 332-338. doi:10.1111/j.1399-6576.2011.02590.x

Calland

J. F.

Turrentine

F. E.

Guerlain

Bovbjerg

Poole

G. R.

Lebeau

Adams

R. B.

(2011). The surgical safety checklist: Lessons learned during implementation. American Surgeon, 77, Article 1131.

Carney

B. T.

West

Neily

Mills

P. D.

Bagian

J. P.

(2010). Differences in nurse and surgeon perceptions of teamwork: Implications for use of a briefing checklist in the OR. AORN Journal, 91, 722-729.

Centers for Disease Control and Prevention. (2011). Inpatient surgery. Retrieved from http://www.cdc.gov/nchs/fastats/insurg.htm

Cooper

H. M.

(2010). Research synthesis and meta-analysis: A step-by-step approach (Vol. 2.). Thousand Oaks, CA: Sage.

Cooper

H. M.

Hedges

L. V.

Valentine

J. C.

(2009). The handbook of research synthesis and meta-analysis. New York, NY: Russell Sage Foundation.

10.

Cullen

K. A.

Hall

M. J.

Golosinskiy

(2009). Ambulatory surgery in the United States, 2006 (National Health Statistics Report No. 11, Rev.). Hyattsville, MD: National Center for Health Statistics.

11.

De Oliveira

J. G. S.

Chang

Kendall

M. C.

Fitzgerald

P. C.

McCarthy

R. J.

(2012). Publication bias in the anesthesiology literature. Anesthesia and Analgesia, 114, 1042-1048. doi:10.1213/ANE.0b013e3182468fc6

12.

de Vries

E. N.

Dijkstra

Smorenburg

S. M.

Meijer

R. P.

Boermeester

M. A.

(2010). The SURgical PAtient Safety System (SURPASS) checklist optimizes timing of antibiotic prophylaxis. Patient Safety in Surgery, 4, Article 6. doi:10.1186/1754-9493-4-6

13.

de Vries

E. N.

Prins

H. A.

Crolla

R. M. P. H.

den Outer

A. J.

van Andel

van Helden

S. H.

Boermeester

M. A.

(2010). Effect of a comprehensive surgical safety system on patient outcomes. New England Journal of Medicine, 363, 1928-1937. doi:10.1056/NEJMsa0911535

14.

Dwan

Gamble

Ghersi

Ioannidis

J. P. A.

Simes

Williamson

P. R.

Von Elm

(2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PloS One, 3(8), e3081. doi:10.1371/journal.pone.0003081

15.

Fecho

Lunney

A. T.

Boysen

P. G.

Rock

Norfleet

E. A.

(2008). Postoperative mortality after inpatient surgery: Incidence and risk factors. Therapeutics and Clinical Risk Management, 4, 681-688.

16.

Gawande

(2010). The checklist manifesto: How to get things right. New York: Metropolitan Books.

17.

Gawande

Zinner

M. J.

Studdert

D. M.

Brennan

T. A.

(2003). Analysis of errors reported by surgeons at three teaching hospitals. Surgery, 133, 614-621.

18.

Grover

F. L.

Johnson

R. R.

Laurie

Shroyer

Marshall

Hammermeister

K. E.

(1994). The veterans affairs continuous improvement in cardiac surgery study. Annals of Thoracic Surgery, 58, 1845-1851. doi:10.1016/0003-4975(94)91725-6

19.

Haynes

A. B.

Weiser

T. G.

Berry

W. R.

Lipsitz

S. R.

Breizat

A.-H. S

Dellinger

E. P.

Gawande

A. A.

(2009). A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine, 360, 491-499. doi:10.1056/NEJMsa0810119

20.

Haynes

A. B.

Weiser

T. G.

Berry

W. R.

Lipsitz

S. R.

Breizat

A.-H. S.

Dellinger

E. P.

Gawande

A. A.

; Safe Surgery Saves Lives Study Group. (2011). Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Quality & Safety, 20, 102-107. doi:10.1136/ bmjqs.2009.040022

21.

Helmiö

Blomgren

Takala

Pauniaho

S. L.

Takala

R. S. K.

Ikonen

T. S.

(2011). Towards better patient safety: WHO Surgical Safety Checklist in otorhinolaryngology. Clinical Otolaryngology, 36, 242-247. doi:10.1111/j.1749-4486.2011.02315.x

22.

The Joint Commission. (2013). Sentinel event data: Root causes by event type 2004-2012. Retrieved from http://www.jointcommission.org/assets/1/18/Root_Causes_Event_Type_04_4Q2012.pdf

23.

Kearns

R. J.

Uppal

Bonner

Robertson

Daniel

McGrady

E. M.

(2011). The introduction of a surgical safety checklist in a tertiary referral obstetric centre. BMJ Quality & Safety, 20, 818-822. doi:10.1136/bmjqs.2010.050179

24.

Khuri

S. F.

Daley

Henderson

Barbour

Lowry

Irvin

Stremple

J. F.

(1995). The National Veterans Administration Surgical Risk Study: Risk adjustment for the comparative assessment of the quality of surgical care. Journal of the American College of Surgeons, 180, 519-531.

25.

Leape

L. L.

Brennan

T. A.

Laird

Lawthers

A. G.

Localio

A. R.

Barnes

B. A.

Hiatt

(1991). The nature of adverse events in hospitalized patients. New England Journal of Medicine, 324, 377-384.

26.

Lee

Regenbogen

Gawande

A. A.

(2008, December). How many surgical procedures will Americans experience in an average lifetime?: Evidence from three states. Paper presented at the Massachusetts Chapter of the American College of Surgeons 55th Annual Meeting, Boston, MA. Retrieved from http://www.mcacs.org/abstracts/2008/P15.cgi

27.

Lingard

Espin

Whyte

Regehr

Baker

G. R.

Reznick

Grober

(2004). Communication failures in the operating room: An observational classification of recurrent types and effects. Quality & Safety in Health Care, 13, 330-334.

28.

Makary

M. A.

Mukherjee

Sexton

J. B.

Syin

Goodrich

Hartmann

Pronovost

P. J.

(2007). Operating room briefings and wrong-site surgery. Journal of the American College of Surgeons, 204, 236-243. doi:10.1016/j.jamcollsurg.2006.10.018

29.

Makary

M. A.

Sexton

J. B.

Freischlag

J. A.

Holzmueller

C. G.

Millman

E. A.

Rowen

Pronovost

P. J.

(2006). Operating room teamwork among physicians and nurses: Teamwork in the eye of the beholder. Journal of the American College of Surgeons, 202, 746-752.

30.

Moher

Liberati

Tetzlaff

Altman

D. G.

(2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. British Medical Journal, 339, b2535. doi:10.1136/bmj.b2535

31.

Neily

Mills

P. D.

Young-Xu

Carney

B. T.

West

Berger

D. H.

. . . Bagian

J. P.

(2010). Association between implementation of a medical team training program and surgical mortality. Journal of American Medical Association, 304, 1693-1700. doi:10.1001/jama.2010.1506

32.

Paige

J. T.

Aaron

D. L.

Yang

Howell

D. S.

Chauvin

S. W.

(2009). Improved operating room teamwork via SAFETY prep: A rural community hospital’s experience. World Journal of Surgery, 33, 1181-1187.

33.

Paige

J. T.

Aaron

D. L.

Yang

Howell

D. S.

Hilton

C. W.

Cohn

J. I.

Chauvin

S. W.

(2008). Implementation of a preoperative briefing protocol improves accuracy of teamwork assessment in the operating room. American Surgeon, 74, 817-823.

34.

Paull

D. E.

Mazzia

L. M.

Wood

S. D.

Theis

M. S.

Robinson

L. D.

Carney

Bagian

J. P.

(2010). Briefing guide study: Preoperative briefing and postoperative debriefing checklists in the Veterans Health Administration medical team training program. American Journal of Surgery, 200, 620-623. doi:10.1016/j.amjsurg.2010.07.011

35.

Rogers

S. O.

Jr. Gawande

A. A.

Kwaan

Puopolo

A. L.

Yoon

Brennan

T. A.

Studdert

D. M.

(2006). Analysis of surgical errors in closed malpractice claims at 4 liability insurers. Surgery, 140, 25-33.

36.

Rydenfält

Johansson

Odenrick

Åkerman

Larsson

P. A.

(2013). Compliance with the WHO Surgical Safety Checklist: Deviations and possible improvements. International Journal for Quality in Health Care, 25, 182-187. doi:10.1093/intqhc/mzt004

37.

Weiser

T. G.

Haynes

A. B.

Dziekan

Berry

W. R.

Lipsitz

S. R.

Gawande

A. A.

(2010). Effect of a 19-item surgical safety checklist during urgent operations in a global patient population. Annals of Surgery, 251, 976-980.

38.

Wolf

F. A.

Way

L. W.

Stewart

(2010). The efficacy of medical team training: Improved team performance and decreased operating room delays: A detailed analysis of 4863 cases. Annals of Surgery, 252, 477-485.

39.

World Health Organization. (2008). WHO guidelines for safe surgery. Geneva, Switzerland: Author. Retrieved from http://gawande.com/documents/WHOGuidelinesforSafeSurgery.pdf

40.

Young-Xu

Neily

Mills

P. D.

Carney

B. T.

West

Berger

D. H.

Bagian

J. P.

(2011). Association between implementation of a medical team training program and surgical morbidity. Archives of Surgery, 146, 1368-1373. doi:10.1001/archsurg.2011.762

41.

Yuan

C. T.

Walsh

Tomarken

J. L.

Alpern

Shakpeh

Bradley

E. H.

(2012). Incorporating the World Health Organization Surgical Safety Checklist into practice at two hospitals in Liberia. Journal on Quality and Patient Safety, 38, 254-260.