Improving the implementation of the WHO Surgical Safety Checklist in a Sierra Leone teaching hospital: Experiences from a multi-faceted patient safety improvement project

Abstract

BACKGROUND AND OBJECTIVE:

Improving the safety of surgical care is an area of growing interest in Sierra Leona, a low resource setting where health outcomes remain poor. One tool shown to improve outcomes is the WHO Surgical Safety Checklist. We report an approach for optimising adherence in a public referral hospital in Freetown, Sierra Leone.

METHODS:

A multi-faceted series of interventions was proposed. Planning involved service evaluation, discussion with key stakeholders and formation of a working group to develop tailored interventions. Implementation involved reformatting the local checklist, formal classroom and simulation training and introduction of protocols and visual aids. Analysis of impact was performed by prospective observation of safety processes both pre- and post-intervention.

RESULTS:

Following the intervention, there was a significant increase in the total number of intraoperative safety processes performed (38.1% pre vs 73.0% post, p < 0.001). A significant improvement was noted in the performance of airway evaluation, correct administration of antibiotics, marking of the surgical site and preoperative team briefing.

CONCLUSION:

This project describes a series of interventions taken to improve the performance of the WHO Surgical Safety Checklist in a low resource setting. We highlight the importance of local partnership working and stakeholder engagement in implementing successful change.

1. Introduction

Sierra Leone is a low-income country, which, despite the recent advance in healthcare, still ranks 181st out of 189 countries on the latest human development index [1]. As a country with a rapidly growing population of 7,092,113, it is served by only 156 doctors and has some of the world’s worst health outcomes [2]. The country has an average life expectancy at birth of 52 years, a maternal mortality rate of 1360/10,000 and an infant mortality rate of 105/1,000 [3].

As in many developing nations, surgical care remains a particularly neglected area in Sierra Leone. Service evaluation of surgical capacity in Sierra Leone concluded that the infrastructure available was insufficient in ‘every aspect’ to provide safe care, with basic amenities such as running water, oxygen and electricity deficient throughout the country [4]. While work has increasingly focused on improving access to surgery, one of the keys to promoting high-quality care is the promotion of patient safety. Unsafe practice can lead to poor outcomes, with preventable adverse events, the result of system failures rather than unsafe individual practice alone [5].

The matter of safe surgical care is a particular issue at Connaught Hospital, the country’s tertiary surgical referral hospital for both adults and children. Here, postoperative mortality following emergency surgery has been recorded as high as 13.8 deaths/100 operations compared with 4.6 deaths/100 operations in the UK [6,7]. Unpublished local research has highlighted several major weaknesses in the provision of safe surgical care at Connaught. Significant concerns have been raised around the issues of informed consent, marking of the surgical site before procedures and the lack of robust checks of patient identity prior to surgery. Many factors were suggested to be contributing to the lack of a ‘safety culture’ at Connaught, including the hierarchical nature of staff interactions, deficiencies in staff training, heavy staff workload and a perceived ‘blame and shame’ culture.

One tool which has been used to address safety issues such as these is the Surgical Safety Checklist. Introduced by the World Health Organization (WHO) in 2008, the Safe Surgery Saves Lives programme aims to reduce the number of deaths related to surgery worldwide by reinforcing accepted safety practices and improving communication. It addresses important issues such as inadequate anaesthetic safety practices, avoidable surgical infection and poor communication among team members. Research has shown significant reductions in operative morbidity and mortality in all settings across the globe when it is used [8], with the greatest benefits seen in low-middle income settings.

Table 1
Barriers to effective checklist implementation at Connaught Hospital

Practical issues Resources

∙ Absence of key team members at relevant points (e.g. no surgeon at sign out) ∙ Shortage of pulse oximeters due to lack of accountability regarding stock

∙ Lack of training in specific processes (e.g. ∙ Lack of facilities to display routine imaging

instrument count, airway assessment) ∙ Insufficient staff for workload

∙ Confusion over who should lead each section ∙ Lack of marker pens, with surgeons not widely

∙ Poorly designed checklist document carrying them individually

∙ Unfamiliarity with the ASA scoring system

∙ Illegible data entries

∙ Missing patient notes

Attitudes Processes

∙ Lack of engagement from staff, particularly surgical teams ∙ Checklist design omits key safety checks such as the marking of the surgical site and preoperative

∙ Hierarchy in the theatre complex discouraging anaesthetic machine and equipment check

communication ∙ No formal patient identification system

∙ Use of the checklist as a ‘tick box exercise’ ∙ No routine instrument and swab count

∙ Lack of emphasis on safety culture within the hospital ∙ Antibiotics often are given in preoperative holding area with significant subsequent delays

∙ Unwillingness to participate in before entering theatre

multi-professional training ∙No clearly allocated staff to lead each stage of the

∙ Unfamiliarity with routine introductions checklist

during a theatre list (embarrassment, familiarity within small teams) ∙ Sign in routinely performed by theatre nurse without the presence of an anaesthetic practitioner

∙ Perceived ‘blame and shame’ culture ∙ Under-reporting of adverse events limiting the ability to monitor outcomes accurately

Practical issues	Resources
∙ Absence of key team members at relevant points (e.g. no surgeon at sign out)	∙ Shortage of pulse oximeters due to lack of accountability regarding stock
∙ Lack of training in specific processes (e.g.	∙ Lack of facilities to display routine imaging
instrument count, airway assessment)	∙ Insufficient staff for workload
∙ Confusion over who should lead each section	∙ Lack of marker pens, with surgeons not widely
∙ Poorly designed checklist document	carrying them individually
∙ Unfamiliarity with the ASA scoring system
∙ Illegible data entries
∙ Missing patient notes

Attitudes	Processes

∙ Lack of engagement from staff, particularly surgical teams	∙ Checklist design omits key safety checks such as the marking of the surgical site and preoperative
∙ Hierarchy in the theatre complex discouraging	anaesthetic machine and equipment check
communication	∙ No formal patient identification system
∙ Use of the checklist as a ‘tick box exercise’	∙ No routine instrument and swab count
∙ Lack of emphasis on safety culture within the hospital	∙ Antibiotics often are given in preoperative holding area with significant subsequent delays
∙ Unwillingness to participate in	before entering theatre
multi-professional training	∙No clearly allocated staff to lead each stage of the
∙ Unfamiliarity with routine introductions	checklist
during a theatre list (embarrassment, familiarity within small teams)	∙ Sign in routinely performed by theatre nurse without the presence of an anaesthetic practitioner
∙ Perceived ‘blame and shame’ culture	∙ Under-reporting of adverse events limiting the ability to monitor outcomes accurately

A version of the checklist was in use at Connaught Hospital, though significant modifications to the original document had been made, both adding and removing elements. Many barriers were identified as being problematic for effective implementation of the WHO checklist (Table 1), with many similar themes seen across other resource-limited settings [9]. These barriers, coupled with some unique institutional factors, meant that at each stage, there were multiple reasons why the WHO checklist was not conducted in line with best practice and therefore did not achieve its aim of improving communication and safety.

While the benefits of the checklist on safe practice are now clearly established there remains difficulties with successful adoption in low resource countries, with research showing variable compliance with the checklist in such settings. Data regarding optimal, low-cost strategies for implementation remain lacking. With this in mind, we aimed to undertake a quality improvement project with several specific aims:

To facilitate the use of the WHO Safer Surgery Checklist in Connaught operating theatres in a way which aligns better with the ideals and templates of the original WHO proposals.

To ensure that the use of the WHO Safer Surgery Checklist at Connaught is benefitting patients and staff at the hospital.

To work in partnership with the surgical team to facilitate any improvements, maintaining frequent and ongoing communication.

2. Methods

We proposed a multi-faceted series of interventions, which in combination aimed to improve the utilisation and performance of the of the WHO checklist (Table 2).

Table 2
Summary of interventions

Stakeholder engagement

∙ Interviews and discussions with key departmental stakeholders including the head of surgery, head of anaesthesia, theatre matron and chief surgical resident

∙ Formation of a working group involving senior staff from all key staffing groups and areas. An initial forum to identify issues was held with subsequent regular meetings to discuss proposed and implemented changes

Checklist reformat

∙ Reformat of the WHO checklist to mirror the original template ensuring visual cues are maintained, language is context-appropriate and can be photocopied in black-and-white with clarity

∙ Addition of essential hospital checks to the document to ensure the individual hospital needs met without compromising the effectiveness of the original WHO 3-stage approach

∙ Presentation to and sign-off by senior hospital management for use in the theatre complex

Staff training

∙ Delivery of multidisciplinary training sessions for perioperative staff

∙ A mix of classroom training and in-situ simulated scenarios

∙ Individual discussion with key local champions

Protocols and materials

∙ Formalised quick reference protocol explaining how the checklist should be conducted in theatres at Connaught

∙ Large colour posters of the reference guide displayed in selected areas (surgical wards, theatres, recovery, holding areas)

∙ Pens available for surgical site marking

A critical approach was to work continually with the surgical department at Connaught Hospital, ensuring their specific individual needs were met, and a tailored approach to interventions considered. Frequent and ongoing communication with key stakeholders was deemed to be vital if changes were to be facilitated and sustained. In addition to individually engaging key stakeholders, a working group was formed containing core staff involved in the perioperative journey. This created a forum with which to freely discuss barriers to optimal implementation and potential interventions to improve this. Connaught Hospital was chosen due to its existing links with the King’s Sierra Leone Project (KSLP). Multiple projects have been performed in partnership with the hospital resulting in a strong working relationship between the organisations.

Fig. 1.

Reformatted institutional checklist.

Designing a checklist which maintained the original WHO objectives while adapting to meet the individual needs of the department was considered vital to the successful adoption of the project [10]. In a process led by the theatre matron, we created a two-sided checklist (Fig. 1). One side mirrored the evidence-based three-phase approach of the original WHO checklist and the second side incorporated the essential perioperative checks requested by the surgical department. We aimed for the modified checklist to maintain key visual cues, to be read clearly and maintain functionality when printed and photocopied in black and white. Language was adapted to ensure it was contextually appropriate and easily understood by all parties using it. Prior to implementation, the checklist was reviewed and agreed by hospital management.

Comprehensive training on the rationale and optimal utilisation of the WHO checklist was delivered. This was scheduled in the period prior to the checklist re-introduction, aiming to maximise the impact and engagement amongst all levels of staff. Training sessions were multidisciplinary in nature, using an approach shown to be successful in the implementation of the WHO checklist in other studies. Training included a combination of classroom teaching and theatre delivered simulation. An initial session was delivered explaining the rationale behind the WHO checklist for improving safer surgery and its evidence of improved outcomes. This was followed by in-situ low fidelity simulation covering a range of theatre cases with which the checklist could be used to identify errors and prevent potential adverse events. Staff were given feedback on communication, and there was an opportunity for group discussion. Staff trained included nurse anaesthetists, theatre nurses, surgical ward nurses, recovery nurses and surgeons.

Fig. 2.

Reference guide displayed in the theatre complex.

Fig. 3.

Alcohol-based pens placed in preoperative patient area.

Accompanying the checklist re-introduction, a formalised quick reference protocol was developed with advice on how to conduct each stage of the checklist in the theatre complex. This was developed in conjunction with the working group and formally agreed by senior staff. It was printed off as large colour posters and sited at key visible locations to act as a learning tool and aide-memoire (Fig. 2). Also, we ensured that alcohol-based marker pens were available in communal areas in designated, labelled holders (Fig. 3).

Initial evaluation of checklist adherence was performed and repeated following the introduction of all interventions. This was achieved through a prospective audit of ‘process measures’, used as a surrogate assessment of successful checklist implementation. The WHO identifies seven process measures in its implementation manual [11] (Table 3). This offers a way of tracking adherence to checklist activities and processes, rather than just measuring checklist document completion post-operatively.

Table 3

Safety processes identified by WHO

1.Marking of the operative site by the surgeon

2.Use of pulse oximetry throughout the administration of anaesthesia in all cases

3.Objective evaluation of the airway

4.Administration of prophylactic antibiotics within one hour before skin incision

5.Verbal confirmation of patient, site and procedure immediately before incision with all team members present

6.Preoperative team briefing to discuss clinical concerns, operative plan, and other critical issues

7.Postoperative team debriefing to discuss problems during the case and concerns for recovery and management of the patient

Cases were observed from arrival to the theatre complex to departure from the operating theatre. We observed cases across a range of theatres and specialities, both pre and post-intervention during two-week periods. Cases were chosen pragmatically based upon the availability of observers. For each case, we recorded the verbal and documented performance of each of the three stages as well as whether each of the seven safety ‘processes’ was performed.

In line with the WHO guidance, the project was deemed a patient safety activity, aiming to improve patient care within a single institution to align with well recognised international protocols. Therefore, as it posed no risk to those patients involved, formal ethical approval was not sought, though permission was sought from the hospital lead [12]. The guidance provided through SQUIRE was used as a framework for writing the manuscript of this quality improvement project [13]. Data were analysed in Microsoft Excel using T-tests and two proportion Z-tests.

Table 4

Analysis of pre- and post-intervention cases

	Cases (Pre-intervention period, n = 18)	%	Cases (Post-intervention period, n = 18)	%	Unadjusted p-value
Demographics
Age (mean, range)	35.0 (2–70)	n/a	23.2 (0–60)	n/a	0.09
Female	12	66.7	10	55.6	0.56
Case-mix
General surgery	10	55.6	12	66.7	n/a
Orthopaedic surgery	7	37.0	1	5.6	n/a
Paediatric surgery	1	5.6	5	27.8	n/a
Intraoperative safety process measures	48	38.1	92	73.0	<0.001
Objective evaluation of the airway performed by the anaesthetic practitioner	6	33.3	12	66.7	0.04
Marking of the operative site performed by the surgeon	0	0	11	61.1	<0.001
Use of pulse oximetry throughout the duration of anaesthesia	18	100	17	94.4	0.29
Prophylactic antibiotics administered within 1-hour prior to skin incision	10	55.6	18	100	<0.001
Confirmation of patient, site and procedure before incision	11	61.1	15	83.3	0.14
Preoperative team briefing performed to discuss concerns with the case	3	16.7	16	88.9	<0.001
Postoperative team debriefing performed to discuss problems with the case and concerns for recovery	0	0	3	16.7	0.08
Checklist stages
Performed aloud	24	44.4	29	53.7	0.21
Documented	30	55.5	41	75.9	<0.001
Sign in
Performed aloud	10	55.6	8	44.4	0.44
Documented	12	66.7	18	100	0.01
Time out
Performed aloud	14	77.8	18	100	0.01
Documented	12	66.7	18	100	0.03
Sign out
Performed aloud	0	0	3	16.7	0.07
Documented	6	33.3	5	27.8	0.74

3. Results

Data from 18 pre-intervention and 18 post-intervention cases were analysed (Table 4). Demographic characteristics of mean age, gender and case-mix were similar between the pre- and post-intervention groups. Overall, following our multi-faceted intervention, there was a significant increase in the total number of intraoperative safety process measures that were performed (38.1% pre vs 73.0% post, p < 0.001). Four of the individual processes showed significant improvement in the rate of performance (Fig. 4). Of note, rates of correct administration of antibiotics almost doubled in the post-intervention group (55.6% pre vs 100.0% post, p < 0.001), and marking of the surgical site increased to 61.1% in the post-intervention group which occurred in none of the patient’s pre-intervention (p < 0.001).

Fig. 4.

Completion of seven safety ‘processes’ pre- and post- intervention.

The increase in performance of safety processes coincided with the increased use of the WHO checklist. Improvements in both verbal and written checklist performance were noted though this did not reach statistical significance for verbal performance (Fig. 5) The highest increase in checklist adherence was seen in the ‘time-out’ phase before incision, with both verbal performance (pre 77.8% vs post 100%, p = 0.01) and documented performance (pre 66.7% vs post 100%, p = 0.03) improving significantly.

4. Discussion and conclusion

Optimising compliance with the WHO checklist can be challenging in all settings but has shown to be important in achieving better surgical outcomes. Analysis of the checklist used at Connaught Hospital has demonstrated many of the issues, where, despite prior introduction, adherence to the checklist remained poor. Strategies for implementing change in low resource-settings are well documented and include the use of educational meetings, interprofessional teaching and tailored interventions [14]. However, detailed reports of how these principles can be used to successfully implement the use of the WHO checklist in low resource setting remain limited [15–17].

Our results show that in spite of the challenges associated with working such a setting, the performance of the WHO safer surgery checklist can be improved through the use of low cost-interventions and engagement with the staff. Our project details specific interventions and strategies used and demonstrates the importance of adapting to meet local needs. A particularly important part of our project was the widespread engagement with the surgical department and key individuals involved in the perioperative pathway. Our project takes the approach of long-term interagency partnership working, which we hope will lead to sustainable improvements. Global health partnerships have been repeatedly shown to contribute to effective and efficient health systems [18]. As well as increasing ownership, ensuring key personnel from Connaught Hospital were involved in tailoring interventions also meant that any changes were made with the local work environment in mind. Interventions tailored to address the specific local barriers have been shown to significantly improve professional practice [14].

Fig. 5.

Performance of the stages of the surgical checklist pre- and post-intervention.

In this project, we used key ‘safety processes’ to measure adherence to the principles of safer surgery aligned to the WHO checklist. Following our multi-faceted series of interventions, we observed a 91.6% relative increase in the total number of these processes performed. We believe that this will lead to better patient outcomes during the perioperative period. Measuring processes rather than relying on checklist compliance was vital to accurately reflect improvements in safer surgical practice. Completion of a paper or verbal checklist does not guarantee that the expected steps have been appropriately followed. Indeed, some discrepancy was noted in our project between improvements in processes performed and the observed checklist completion. While significant increases in documented and verbal performance of the ‘time-out’ section were observed, this was not true for the ‘sign-in’ and ‘sign-out’ sections.

There are several reasons why we may have observed this yet observe an overall increase in safety processes performed. Improved attitudes and increased consideration towards surgical safety may have contributed, given the training delivered and increased awareness generated by the interventions. It has been demonstrated that the introduction of the WHO checklist alone can improve perceptions of teamwork and safety climates within a department. In our project, significantly improved performance of the pre-incision ‘time-out phase’ was noted post-intervention, with the verbal performance of this observed in 100% of cases. Four of the seven safety processes recorded are checked at this stage, and thus it may explain the improvement in these. While there was a slight decrease in performance of the ‘sign-in’ stage, significant changes in how this was conducted may explain why increased safety processes were performed. Previously this had been undertaken by the ward and theatre nurse in combination, who may fail to act on checks related to anaesthesia. By involving the anaesthetic team, a higher number of processes may have been highlighted as requiring completion. Conduct of the postoperative ‘sign-out’ phase remained poor despite the interventions and was reflected in a low number of postoperative briefings performed. Reasons for this are unclear but may relate to the decreased presence of senior personnel at this stage of the procedure or a perceived lack of importance of these checks. The low engagement with the ‘sign-out’ relative to the previous sections is consistent with other data from both high and low-resource settings [19].

This project represents an observational experience from a single centre, and as such, there are several limitations. A small sample size reduces the ability to interpret results. Lack of robust reporting on outcomes and complications prevents determination on whether the improvement in safety processes has led to material differences to the patient. No control data was collected for comparison, though we have no reason to believe that any other intervention during the study period affected our outcomes. In addition, observer bias may have contributed to outcomes though staff where unaware which cases were audited, and we believe this is unlikely to have significantly impacted results.

The strength of this project, however, is the demonstration that the use of local engagement and low-cost interventions can be used to significantly improve safe surgical care in a publicly funded large government hospital. We believe that by sharing analysis of the issues faced at Connaught Hospital, and steps taken to overcome these when implementing the WHO checklist will be useful for similar institutions looking to improve surgical safety. We hope to engage the department going forward and ensure that there is continuous evaluation of checklist use and its optimal implementation. Continuation of the working group will be key to ensuring this. We believe if sustainable change is to be achieved. This must be led by the operating department with support from allied partners.

Footnotes

Acknowledgements

The development of this article has been supported by the Kings Sierra Leone Partnership (KSLP) with funding assistance from the Africa Grants Project (AGP). The opinions herein are those of the authors and do not necessarily reflect the views of KSLP or AGP.

Conflict of interest

The authors confirm that they do not have any of the below associations to declare:

(1) Associations with commercial entities that provided support for the work reported in the submitted manuscript (the timeframe for disclosure in this section of the form is the lifespan of the work being reported).

(2) Associations with commercial entities that could be viewed as having an interest in the general area of the submitted manuscript (in the 36 months before submission of the manuscript).

(3) Any similar financial associations involving authors’ spouses, partners, their children under 18 years.

(4) Non-financial associations that may be relevant to the submitted manuscript.

Informed consent

The authors confirm that in line with the WHO guidance, the project was deemed a patient safety activity, aiming to improve patient care within a single institution to align with well recognised international protocols. Therefore, as it posed no risk to those patients involved, formal ethical approval was not sought, though permission was sought from the hospital lead.

References

United Nations Development Programme. Human Development Report 2019. 2019. 978-92-1-126439-5.

World Health Organisation. Sierra Leone Annual Report: A Year in Focus 2017. 2017.

Sierra Leone. World Health Organisation. [Online]. Available from: https://www.who.int/countries/sle/en/.

Kingham

Kamara

Cherian

. Quantifying surgical capacity in Sierra Leone: A guide for improving surgical care. Arch Sur. 2009;144:122–7.

Kizer

Stegun

. Volume 4: Programs, Tools, and Products. Advances in Patient Safety: From Research to Implementation. s.I.. Agency for Healthcare Research and Quality; 2005.

Chima

Anesthesia practice and perioperative outcomes at two tertiary care hospitals in Freetown Sierra Leone. Anaesthesia & Analgesia. 2019;123:213–27.

Ozdemir

Mortality of emergency general surgical patients and associations with hospital structures and processes. British Journal of Anaesthesia. 2016;116:54–62.

Haynes

Weiser

Berry

Lipsitz

Breizat

A-HS

Patchen Dellinger

A Surgical Safety Checklist to reduce morbidity and mortality in a global population. The New England Journal of Medicine. 2016;360:491–9.

Woodman

Walker

. WHO Surgical Safety Checklist. World Federeration of Societies of Anaesthesiologists; 2016.

10.

Vivekanantham

Surgical Safety Checklists in developing countries. International Journal of Surgery. 2014;12:2–6.

11.

World Health Organization. Implementation Manual: WHO Surgical Safety Checklist 2009 2009.

12.

World Health Organisation. Ethical Issues in Patient Safety Research 2013.

13.

Ogrinc

Davies

Goodman

Batalden

Davidoff

Stevens

. (Standards for quality improvement reporting excellence): Revised publication guidelines from a detailed consensus process. BMJ Quality and Safety. 2016;25:986–92.

14.

Pantoja

Opiyo

Lewin

Paulsen

Ciapponi

Wiysonge

Herrera

Rada

Peñaloza

Dudley

Gagnon

Garcia Marti

Oxman

. Implementation strategies for health systems in low-income countries: An overview of systematic reviews (Review). Cochrane Database of Systematic Reviews. 2017;9:1–113.

15.

Yuan

Walsh

Tomarken

Alpern

Shakpeh

Bradley

. Incorporating the World Health Organization Surgical Safety Checklist into practice at two hospitals in Liberia. Jt Comm J Qual Patient Saf. 2012;38:254–60.

16.

Kwok

Funk

Baltaga

Lipsitz

Merry

Dziekan

Implementation of the World Health Organization Surgical Safety Checklist, including introduction of pulse oximetry, in a resource-limited setting. Ann Surg. 2013;257:633–9.

17.

Bashford

Reshamwalla

McAuley

Allen

McNatt

Yohannes

. Implementation of the WHO Surgical Safety Checklist in an ethiopian referral hospital. Gebremedhen Patient Safety in Surgery. 2014;8:16.

18.

THET. THET Health Partnership Scheme Frequently Asked Questions. [Online]. 2017. Available from: http://www.thet.org/health-partnership-scheme/health-partnerships.

19.

Cadman

. Use of the WHO Surgical Safety Checklist in low and middle income countries: A review of the literature. Journal of Perioperative Practice. 2014;28:334–8.