Effect of Compliance With a Nurse-Led Intensive Care Unit Checklist on Clinical Outcomes in Mechanically and Nonmechanically Ventilated Patients

Abstract

Background:

Use of checklists brings about improvements in a variety of patient outcomes. Nevertheless, whether compliance with a nurse-led intensive care unit (ICU) checklist produces the same effect is currently unknown.

Methods:

This is a retrospective analysis of data obtained during the implementation of a quality improvement project consisting of the utilization of a nurse-led ICU checklist. A consecutive series of checklists obtained from patients admitted in our ICU during 7 consecutive months were included. The ICU stay, hospital stay, and ventilator stay were compared between patients whose practitioners completed or did not complete the checklist. Variables were analyzed using Mann-Whitney U tests for continuous variables and Fisher exact tests for categorical variables. A 2-tailed P < .05 was considered statistically significant.

Results:

One thousand checklists, corresponding to 346 eligible patients, were collected over 7 months. Mechanical ventilation was used in 203 (59%) patients. Completed checklists were observed for 37.6% (n = 130) of all patients and 38.9% (n = 79) of mechanically ventilated patients. After adjusting for age, Acute Physiology and Chronic Health Evaluation II (APACHE II), body mass index, reason for admission, and type of ICU, completion of the checklist was associated with a 20% increase in the number of days in the ICU compared with the group with incomplete lists. In mechanically ventilated patients, completion of the checklist was associated with a 31% increase in hospital length of stay, a 34% increase in the number of ICU days, and a 32% increase in mechanical ventilation days.

Conclusion:

Compliance with completion of a nurse-led ICU checklist was associated with prolonged ICU stay, hospital stay, and ventilator stay.

Keywords

safety quality checklist

Introduction

Checklists have been utilized as important tools to achieve systematization of processes in a variety of industries, such as aviation, engineering, and medicine. Due to the increasing complexity seen in the intensive care unit (ICU) environment and growing concerns in regard to patient safety, the implementation of checklists has become essential to assure a homogeneous delivery of care. Several experiences were previously described, in which the use of checklists brought about improvement in patient outcomes, such as a reduction in catheter-related bloodstream infection,^1,2 decrease in mechanical ventilation weaning time,³ and higher compliance with clinical practice guidelines.⁴ Furthermore, when the presence of an ICU checklist was associated with prompting physicians addressing its utilization, the rate of compliance with a variety of other processes of care improved.⁵ Specifically, days on empiric antibiotic therapy and central venous catheters decreased, and the rates of deep venous thrombosis and stress ulcer prophylaxis increased. Based on the aforementioned data, we developed a nurse-led ICU checklist and underwent a quality improvement (QI) project to optimize processes of care. The goal of this study is to assess whether nursing compliance with filling an 18-item ICU checklist is associated with improvement in clinical outcomes in mechanically and nonmechanically ventilated ICU patients.

Materials and Methods

This is a retrospective analysis of prospectively collected data obtained during the implementation of a QI project consisting of the utilization of a newly developed ICU checklist. It was exempt from the Creighton University Medical Center (CUMC) institutional review board.

A Nursing ICU checklist was implemented in the ICU at CUMC on December 2012 (Figure 1). Once patients were admitted in the ICU, nurses completed the ICU checklists for each patient on a daily basis, at 6 pm. Therefore, each patient had an ICU checklist completed every 24 hours. The aforementioned checklist was also used as a communication tool between nurses, as it was checked out to the incoming nurse on every shift (twice daily, at 6 am and 6 pm).

Figure 1.

Nurse-led intensive care unit (ICU) checklist.

A consecutive series of patients admitted to the ICU between December 2012 and June 2013 were included in this study. All daily ICU checklists were collected at the end of the year and audited for compliance with their completion. Patients were divided into 2 groups based on the presence of completed or noncompleted checklists. We defined “complete checklists” as those checklists in which every single item was reviewed (checked). Checklists with at least 1 item incomplete were deemed “incomplete checklist.” For those patients who had an ICU admission longer than 24 hours, and therefore more than 1 filled checklist, we considered there was compliance with checklist completion if at least two-third of the total number of checklists for a particular patient were complete. Demographic data, the Acute Physiology and Chronic Health Evaluation II (APACHE II) score, reason for ICU admission, and type of ICU in which patients were admitted (, medical, trauma, etc) were collected by 2 investigators (HSA and ASA.). We compared the hospital and ICU lengths of stay between patients with complete and patients with incomplete checklists. Furthermore, we assessed the previously mentioned outcomes plus ventilator days in a subgroup of mechanically ventilated patients.

Statistical Analysis

Continuous demographic and clinical variables are presented as mean ± standard deviation, whereas categorical variables are presented as frequency and percentage. Differences in these variables between patients whose practitioners completed or did not complete the checklist were evaluated using Mann-Whitney U tests for continuous variables and Fisher exact tests for categorical variables.

Two groups of patients were considered for the analysis—all patients and mechanically ventilated patients. The outcomes for all patients included length of hospital stay and days in the ICU; outcomes for mechanically ventilated patients included the same 2 outcomes with the addition of days on mechanical ventilation.

All outcomes were treated as count variables. Count variables are generally modeled using either Poisson or negative binomial regression. Poisson regression assumes the model predicted mean rate and variance are equal, which is often an untenable assumption as the variance is often greater than the mean (known as overdispersion).⁶ Overdispersion is quantifiable using a negative binomial regression model that includes a scale parameter allowing the variance to increase as needed independent of the mean. When the scale parameter equals 0, the Poisson and negative binomial models produce identical results; thus, the Poisson model is nested within the negative binomial model allowing for direct comparison of model fit via likelihood ratio test. For all outcomes in our study, the negative binomial model fits significantly better than the Poisson model. The independent variable of interest was completed versus incomplete checklists, as a result, only the effect of this variable will be interpreted.

All models adjusted for demographic and clinical covariates that were theoretically related to the outcomes. Covariates included age, APACHE-II score, body mass index (BMI), reason for admission, and type of ICU. For analysis, continuous covariates were centered near their grand mean (ie, age = 55, APACHE-II = 13, and BMI = 28).

All analyses were conducted using SAS v 9.3 (SAS Institute, Inc, Cary, North Carolina); a 2-tailed P < .05 was considered statistically significant.

Results

Over a 7-month period, a total of 1000 checklists, corresponding to 349 ICU patients, were collected. Three patients had missing data (2 admitting diagnoses and 1 BMI). Complete data were observed for 346 patients of which 203 (59%) were mechanically ventilated. Completed checklists were observed for 37.6% (n = 130) of all patients and 38.9% (n = 79) of mechanically ventilated patients. Differences in the demographic and clinical variables for all participants, stratified by completion of the nursing checklist, are presented in Table 1.

Table 1.

Checklist Completed Versus Not Completed Differences in Demographic and Clinical Variables.

	Completed (n = 130)	Not Completed (n = 216)	P
Age	57.33 ± 18.32	54.88 ± 18.28	.244
APACHE-II	13.89 ± 7.40	13.62 ± 7.61	.600
Body mass index	27.85 ± 7.27	29.09 ± 8.43	.349
Length of hospital stay	14.30 ± 17.00	13.85 ± 16.77	.928
Days in ICU	7.27 ± 6.63	7.99 ± 14.58	.625
Days mechanically ventilated^a	6.79 ± 9.45	7.44 ± 16.12	.812
Reason for admission
Sepsis/septic shock	11 (9.8)	18 (7.7)	.536
Respiratory failure	14 (12.5)	26 (11.1)	.721
Trauma	20 (17.9)	47 (20.1)	.665
Toxicity	7 (6.3)	28 (12.0)	.127
Central nervous system	11 (9.8)	24 (10.3)	1.000
Bleeding other than brain	10 (8.9)	13 (5.6)	.254
Cardiovascular causes	7 (6.3)	5 (2.1)	.062
Postoperative observation	32 (28.6)	73 (31.2)	.708
Type of ICU
Medical	58 (51.8)	97 (41.5)	.083
Surgical	30 (26.8)	73 (31.2)	.452
Trauma	19 (17.0)	45 (19.2)	.659
Neurosurgery	5 (4.5)	19 (8.1)	.262

Abbreviations: APACHE II, Acute Physiology and Chronic Health Evaluation II; ICU, intensive care unit.

^a Only includes patients who were mechanically ventilated (N = 203; completed n = 61 and not completed n = 142).

For all patients, completion of the nursing checklist was associated with a statistically significant increase in the number of ICU days (see Table 2); completion was not associated with length of hospital stay. After adjusting for age, APACHE-II score, BMI, reason for admission, and type of ICU, completion of the nursing checklist was associated with an average 20% increase in the number of days in the ICU compared to not completing the nursing checklist.

Table 2.

Adjusted Relative Ratios (95% CI) for Hospital Length of Stay, Days in ICU, and Days Mechanically Ventilated for Mechanically Ventilated Patients.^a

	All Patients (N = 346)	Mechanically Ventilated Patients (n = 203)
	Days in ICU	Hospital Length of Stay	Days in ICU	Mechanically Ventilated Days
Intercept	5.03 (3.26-7.75)	10.26 (5.31-19.85)	5.82 (2.90-11.67)	2.99 (1.24-7.24)
Age (0-55)	1.00 (0.99-1.00)	1.00 (0.99-1.01)	1.00 (0.99-1.01)	1.00 (0.99-1.01)
APACHE-II (0-13)	1.05 (1.04-1.07)	1.03 (1.02-1.05)	1.04 (1.02-1.05)	1.05 (1.03-1.07)
Body mass index (0-28)	1.00 (0.99-1.01)	1.00 (0.98-1.01)	1.00 (0.99-1.01)	1.00 (0.99-1.02)
Reasons for admission
Postoperative observation	1.00	1.00	1.00	1.00
Sepsis/septic shock	2.60 (1.67-4.04)	2.03 (1.19-3.45)	3.50 (2.00-6.12)	4.28 (2.25-8.12)
Respiratory failure	2.28 (1.54-3.37)	1.51 (0.98-2.32)	2.40 (1.52-3.79)	3.52 (2.05-6.03)
Trauma	1.64 (0.99-2.69)	1.77 (0.94-3.34)	2.10 (1.11-3.97)	3.02 (1.32-6.91)
Toxicity	1.32 (0.85-2.05)	1.71 (0.97-3.01)	1.77 (0.98-3.19)	1.92 (0.94-3.94)
Central nervous system	1.42 (0.92-2.18)	1.23 (0.67-2.28)	1.57 (0.82-2.99)	2.63 (1.23-5.64)
Bleeding other than brain	0.91 (0.56-1.50)	0.92 (0.42-1.98)	1.00 (0.44-2.29)	1.20 (0.44-3.23)
Cardiovascular causes	2.49 (1.45-4.26)	2.90 (1.39-6.05)	4.24 (1.99-9.03)	7.51 (3.02-18.70)
Type of ICU
Neurosurgery	1.00	1.00	1.00	1.00
Medical	0.60 (0.40-0.89)	0.90 (0.47-1.71)	0.56 (0.28-1.12)	0.50 (0.21-1.21)
Surgical	1.14 (0.74-1.75)	1.21 (0.63-2.33)	1.04 (0.52-2.08)	1.13 (0.67-2.73)
Trauma	0.87 (0.56-1.35)	0.88 (0.46-1.67)	0.72 (0.40-1.32)	0.68 (0.33-1.39)
Checklist completed	1.20 (1.01-1.43)	1.31 (1.06-1.62)	1.33 (1.07-1.66)	1.32 (1.01-1.73)

Abbreviations: APACHE II, Acute Physiology and Chronic Health Evaluation II; CI, confidence interval; ICU, intensive care unit.

^aRatios in bold font are statistically significant at P < .05, also indicated by 95% CIs that do not include 1.00. The intercept value represents the rate when all other variables equal 0. All rate ratios within a given column are statistically adjusted for the other effects in the column.

For mechanically ventilated patients, completion of the nursing checklist was associated with statistically significant increases in all 3 outcomes (see Table 2). Specifically, after adjusting for age, APACHE-II score, BMI, reason for admission, and type of ICU, completion of the nursing checklist was associated with a 31% increase in hospital length of stay, a 34% increase in the number of ICU days, and a 32% increase in mechanically ventilated days, on average, compared to patients whose nursing checklist was not completed.

Finally, we evaluated the specific reasons why the checklist was not completed for all 216 patients (see Table 3). Although most reasons ranged between 2.3% and 7.9%, the 2 primary reasons for not completing the checklist were sepsis screening and sepsis.

Table 3.

Reasons for Not Completing Nursing Checklist (n = 216).

	n (%)
Sedation	7 (3.2)
Analgesia	10 (4.6)
Delirium	14 (6.5)
Deep vein thrombosis	11 (5.1)
Skin integrity	16 (7.4)
Mobility	9 (4.2)
Restraints	6 (2.8)
Fall risk	12 (5.6)
Nutrition	5 (2.3)
Sepsis screening	189 (87.5)
Sepsis	185 (85.6)
Glucose control	12 (5.6)
Mechanical ventilation	9 (4.2)
Head of bed	14 (6.5)
Stress ulcer prophylaxis	17 (7.9)
Foley	12 (5.6)
IV access	6 (2.8)

Abbreviation: IV, intravenous.

Discussion

The present study shows the following results: (1) compliance with the utilization of an 18-item, nurse-led ICU checklist is associated with an increase in the ICU stay; (2) the application of the aforementioned checklist only to mechanically ventilated ICU patients is associated with prolonged stay on mechanical ventilation and ICU and hospital lengths of stay.

Omission errors impose a great risk on patient safety, as they usually pass unnoticed and are frequently underreported.⁷ In fact, one study demonstrated that patients receive only about 50% of recommended evidence-based therapy.⁸ Due to the increased complexity and multiple comorbidities found in ICU patients, they represent a vulnerable population, which is greatly exposed to these types of errors. As a matter of fact, a review suggested that 167 819 lives could be saved in the ICU setting by eliminating errors of omission.⁹ Consistent with these findings, it has been shown that severity of critical illness is inversely related to the likelihood of receiving “best practice” therapeutics, with each 1% increase in APACHE II score resulting in a 1% decreased chance of “best practices” being implemented.¹⁰ In an attempt to minimize omission errors in our ICU, we developed a nursing checklist comprising 18 items to be addressed on a daily basis. Despite our attempt, implementation and compliance with the checklist were not associated with improvements in specific clinical outcomes, such as ICU, hospital, and ventilator lengths of stay. Conversely, the opposite results were found.

Several factors could have contributed to these results. First, as readmission rate was not collected in our study, it is unknown whether patients with shorter ICU and/or hospital lengths of stay had, in fact, higher rate of ICU readmission or not. Therefore, depending on the distribution of readmission rate between both the groups, a prolonged ICU stay, hospital stay, and mechanical ventilator stay with the use of our checklist could have been a sign of optimal patient care rather than the opposite. Second, our study did not assess the impact of behavioral interventions (ie, physician prompting), as we only evaluated compliance with filling the checklist itself. Hence, it is possible that the mere completion of a long checklist without any related behavioral intervention or action plan may have delayed ICU throughput until all items were completed. Third, it is possible that nurses participating in this QI project presented checklist “fatigue.” The overwhelming number of required checkboxes may have become a burden rather than an aid for the nurses who may have unattended other aspects of patient care in order to fulfill the requirements of the checklist. In fact, it is not surprising that “sepsis screening,” which includes 15 items to be reviewed, was the most common reason for incomplete checklist. Consequently, a prolonged ICU stay, hospital stay, and mechanical ventilator stay may be a reflection of nursing fatigue rather than lack of checklist effectiveness. It is worth mentioning that our study implemented a checklist that includes 18 items, with each of them representing current standards of care. To the best of our knowledge, this is the lengthiest ICU checklist when compared with the others described in previous studies. Specifically, Cheema and colleagues¹¹ implemented a ventilator-associated pneumonia (VAP) prevention bundle that consisted of 5 items. Further, feedback from staff involved in that study suggested that an excessive documentation burden associated with the checklist may have contributed to a “checklist burnout” and resulted in a “checklist washout phase,” in which a rebound in the VAP incidence was observed after a significant initial decrease. As our study was conducted over a period of 7 months, it is likely that this type of burnout had occurred.

Importantly, our study presents numerous strengths. We implemented a checklist in a combined ICU, which includes medical, surgical, trauma, and neurosurgical patients. Therefore, the results of our study could possibly be extrapolated to other ICUs with similar design. Second, our checklist included 18 processes of care. To our knowledge, this is the most comprehensive ICU checklist and the only one developed and implemented by nurses. Third, as we retrospectively evaluated prospectively collected checklists (rather than prospective observation), it is unlikely that our results were altered by Hawthorne effect that is usually seen during periods of observation.

Despite the aforementioned strengths, several limitations do exist in our study. Particularly, the study was conducted in a single institution and included only 346 patients, limiting its generalizability. Also, the retrospective nature of our study predisposed our analysis to selection and information biases. Specifically, it is possible that some patients admitted in the ICU during the included period of time did not have an ICU checklist performed. Therefore, only patients with presence of checklists were included, incurring on selection bias. Similarly, we assumed accuracy of collected data in terms of demographics, outcomes, and other variables. Nevertheless, it is possible that some information could have been inaccurate or missing, incurring in informational bias.

Summary/Key Points

In conclusion, this single-site study demonstrated that compliance with completion of a comprehensive nurse-led ICU checklist was associated with prolonged ICU stay. Furthermore, the application of the checklist to mechanically ventilated patients was associated with increase ICU, hospital, and ventilator lengths of stay. Nevertheless, lack of balance outcomes, such as readmission rate, precludes us to conclude that ICU checklists worsen clinical outcomes. In fact, further studies in this topic may provide us a better understanding of the role and effectiveness of ICU checklists in patient care.

Footnotes

Authors’ Note

Haitham S. Al Ashry has made substantial contributions to acquisition and interpretation of data, has drafted the submitted article, has provided final approval of the version to be published, and has agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Ahmed S. Abuzaid has made substantial contributions to acquisition and interpretation of data, has provided final approval of the version to be published, and has agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Ryan W. Walters has made substantial contributions to conception and analysis and interpretation of data, has drafted the submitted article, has provided final approval of the version to be published, and has agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Ariel M. Modrykamien has made substantial contributions to conception and design and analysis and interpretation of data. He has drafted the submitted article and revised it critically for important intellectual content, has provided final approval of the version to be published, and has agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Berenholtz

Pronovost

Lipsett

. Eliminating catheter-related bloodstream infections in the intensive care unit. Crit Care Med. 2004;32(10):2014–2020.

Pronovost

Needham

Berenholtz

. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med. 2006;355(26):2725–2732.

Walsh

Dodds

McArdle

. Evaluation of simple criteria to predict successful weaning from mechanical ventilation in intensive care patients. Br J Anaesth. 2004;92(6):793–799.

DuBose

Inaba

Shiflett

. Measurable outcomes of quality improvement in the trauma intensive care unit: The impact of a daily quality rounding checklist. J Trauma. 2008;64(1):22–27.

Weiss

Moazed

McEvoy

. Prompting physicians to address a daily checklist and process of care and clinical outcomes: A single-site study. Am J Respir Crit Care Med. 2011;184(6):680–686.

Stroup

. Generalized Linear Mixed Models: Modern Concepts, Methods and Applications. Boca Raton, FL: CRC Press; 2012.

Cook

Montori

McMullin

Finfer

Rocker

. Improving patients’ safety locally: Changing clinician behaviour. Lancet. 2004;363(9416):1224–1230.

McGlynn

Asch

Adams

. The quality of health care delivered to adults in the united states. N Engl J Med. 2003;348(26):2635–2645.

Pronovost

Rinke

Emery

Dennison

Blackledge

Berenholtz

. Interventions to reduce mortality among patients treated in intensive care units. J Crit Care. 2004;19(3):158–164.

10.

Ilan

Fowler

Geerts

Pinto

Sibbald

Martin

. Knowledge translation in critical care: factors associated with prescription of commonly recommended best practices for critically ill patients. Crit Care Med. 2007;35(7):1696–1702.

11.

Cheema

Scott

Shambaugh

. Rebound in ventilator-associated pneumonia rates during a prevention checklist washout period. BMJ Qual Saf. 2011;20(9):811–817.