Electronically Triggered Sepsis Alert in Non-Intensive Care Inpatients Using Modified Systemic Inflammatory Response Syndrome Criteria: A Retrospective Observational Study with In-Depth Analysis of Surgical Patients

Patients and Methods

This retrospective observational study included patients more than 18 years old who were admitted to the University of Minnesota Medical Center–Fairview East Bank Campus (UMMC) from August 1, 2015–March 31, 2016. This is an 874-bed, academic, tertiary referral hospital. The sBPA protocol was developed by a multi-disciplinary working group (medical and surgical intensivists, internist, general surgeon, pharmacist, nurse manager, medical informaticists, and a clinical quality manager). Meetings were held biweekly to develop an algorithm to be used throughout the entire Fairview Hospital system. The protocol consisted of an sBPA electronic trigger that responded to abnormal vital signs and laboratory values entered in the electronic medical record (EPIC). An sBPA appeared in EPIC on opening the patient's chart for a nurse (RN) if the patient met at least two mSIRS criteria (heart rate >100 beats/minute, temperature <95° or >101° F, white blood cell count (WBC) <4,000 or >12,000/mL, ventilatory rate (VR) >20 breaths/minute, or systolic blood pressure (SBP) <90 mm Hg). If the patient had an operation performed, for the first 24 hours after surgery, a higher threshold for sBPA triggering was applied requiring three of the mSIRS criteria along with a higher temperature (<95° or >101.5°F). After the sBPA was triggered, the nurse ordered an immediate serum lactate assay without needing a physician order. If the lactate concentration was ≥2 mmol/L, the nurse had a new positive lactate specific sBPA fire, instructing notification of the responsible provider to see the patient. Any provider (physician, nurse practitioner, or physician assistant) who opened the patient's chart received a provider-specific sBPA with instructions to evaluate the patient and use a sepsis bundle order set, including a fluid bolus, blood culture, and antibiotic administration if indicated and to document their findings in a standardized progress note. If the lactate concentration was >4 mmol/L, a “rapid response” was initiated, and consideration for transfer to the intensive care unit (ICU) was recommended in addition to the sepsis order set (Fig. 1 and Appendix). The sBPA does not fire in the ICU, operating room, or post-anesthesia care unit (PACU).

OPEN IN VIEWER

FIG. 1.

Sepsis Best Practice Trigger Alert system.

Our primary endpoint was how frequently and appropriately the sBPA fired in patients having surgery versus those not having surgery. Our secondary endpoints were firing and nonfiring of the sBPA in surgical patients, comparing differences between these groups and their outcomes. Data for patients admitted during the study period were obtained from an internal database developed for tracking sepsis after initiation of the sBPA protocol. Patients admitted to the ICU, inpatient rehabilitation, behavioral medicine, and obstetrics were excluded, as were patients younger than 18 years. An EPIC query was performed to obtain the admission date, discharge date, and sepsis diagnosis (International Classification of Diseases [ICD]-9 and ICD-10 codes, as listed in the Appendix).

Data for surgery patients were obtained from the University of Minnesota Surgical Infection Surveillance database. In this database, all surgical procedures performed in UMMC are recorded prospectively along with patient and hospital characteristics including age, gender, surgical service, wound class (clean, clean-contaminated, contaminated, or dirty), American Society of Anesthesiologists class (ASA 1–6), operative duration, body mass index (BMI), emergency operation (yes/no), admission date, discharge date, presence of surgical site infection (SSI), and death. An SSI was determined according to National Healthcare Safety Network (NHSN) definitions and included both superficial and deep infections [15].

Data preparation and analyses were performed using SPSS version 22 (IBM, Armonk, NY). Surgery patients were compared with patients not having surgery in the number of sBPA triggers, lactate assays, abnormal lactate concentrations, and incidence of sepsis (defined by ICD-9 code) using χ² testing. Specificity, sensitivity, negative predictive value (NPV), and positive predictive value (PPV) were calculated for both groups as well.

All surgery patients were then analyzed separately. They were divided into two groups: Those who triggered the sBPA and those who did not. Patient and procedure characteristics were compared in the two groups using χ² testing. Data are presented as the mean ± standard deviation for normal distribution and median (interquartile range) for skewed data. Significance was set at p < 0.05.

Results

Over the eight-month study period, 10,335 patients were included, 2,158 of whom had surgery. Differences in the number of sBPAs triggered, lactate concentrations measured, and abnormal lactate values in the patients having and not having surgery are shown in Table 1. This includes simultaneous sBPAs to any provider who opens the chart after their corresponding level of alert has been triggered. For instance, if a nurse sBPA has been triggered, then any nurse who enters the patient's chart will see the same trigger; and this is recorded. Similarly, if a physician sBPA has been triggered for a lactate concentration >2 mmol/L, then any physician or mid-level provider who opens the chart sees the sBPA. As a result of the sBPAs, 1,455 blood samples were drawn on surgery patients for lactate measurements, and 7,380 were drawn for patients not having surgery. Of these, 13% compared with 25% of lactate concentrations >2 and 1% vs. 3% of concentrations >4 mmol/L were recorded for patients having and not having surgery, respectively.

The sensitivity and specificity for both groups of patients for an ICD-9/10 sepsis category diagnosis at any time during the hospitalization are shown in Table 2. The group not having surgery had a slightly higher, although still modest, sensitivity at 67%; and the groups had similar specificity. The PPV was higher in the group not having surgery at 25%, and the NPV was high in both groups.

The remaining analysis was performed only on patients having surgery. Table 3 demonstrates a breakdown of the case mix of procedures performed during the study period. This demonstrates a predominance of general, cardiovascular, transplant and neurosurgery. Patient and procedure characteristics were evaluated for patients having an sBPA during an admission and those who did not. As demonstrated in Table 4, roughly one third of the surgery patients triggered an sBPA at some point during their hospitalizations. The median number of alerts per patient was four. There were no significant differences in gender or BMI between the groups; however, the sBPA patients were younger, more likely to have had an emergency operation, and had longer operative times. In addition, more patients having an sBPA had a wound class of 4 (Fig. 2) and were more likely to have been in ASA class 3 or 4.

OPEN IN VIEWER

FIG. 2.

Wound classification distribution between sBPA and no sBPA patients.

Data on monthly lactate assays were evaluated for surgery patients who triggered an sBPA (Fig. 3). Initially, we saw a steady increase in utilization; however, that trend disappeared as the protocol was more widely utilized and blood was drawn approximately 180 times per month. Of the total blood samples drawn, 13% had a >2 mmol/L lactate concentration, and 1% had >4 mmol/L. In fact, only 12 lactate concentrations were found to be >4 mmol/L in the surgery patients outside of the ICU throughout the study duration.

OPEN IN VIEWER

FIG. 3.

Monthly lactate concentration trends from August 2015–March 2016 in surgery patients who fired the sBPA.

Given the higher threshold for triggering the sBPA in the 24-hour post-operative period, we wanted to examine the timing of the first blood draw after surgery. There were only 10 blood samples drawn during this time, with seven of the results being abnormal (Fig. 4). The lactate assays increased considerably in the next two days, with a resulting decrease in the proportion that were abnormal after the threshold went back to requiring only two of the five mSIRs criteria before the sBPA was triggered. The highest incidence of abnormal lactate concentrations was found in the first six days after surgery. Within that period, the highest proportion of abnormal results and lowest number of normal alerts were seen within the first 24 hours. We also noted only three lactate concentrations >4 mmol/L within 24 hours of surgery (two of these patients were already being treated for sepsis) and two on post-operative day 1. The remaining were found more than three days after surgery, with the majority found in the 7- to 30-day time frame.

OPEN IN VIEWER

FIG. 4.

Timing of first blood draw for lactate measurement after surgery and associated percentage distribution of abnormal lactate values.

Next, we compared the outcomes of patients who triggered the sBPA and those who did not. The length of stay was more than twice as long for the sBPA patients, and the proportion of patients with SSIs was more than three times higher. There was no difference in the mortality rate in the groups at 4% vs. 3% (p = 0.15; Table 5).

Discussion

Early recognition of sepsis and timely interventions have a significant impact on patient outcomes [1–6]. Nationally, many institutions are developing early-detection screening systems to aid in prompt diagnosis. Various methods have been utilized; our institution sought to develop an automatic trigger based on the EMR using established criteria with an additional risk stratifier, namely an elevated serum lactate concentration. Jones et al. found that using a nurse-driven screening tool twice daily was able to decrease the inpatient death rate attributable to sepsis by 9% and reduce healthcare costs [6]. Moore et al. created a sepsis screening tool in their surgical ICU that used bedside nurses as the first step in screening, reducing the mortality rate from sepsis by one third. These investigators also found that among nurses, awareness of the signs of sepsis was low and that, not surprisingly, they already had significant and conflicting demands on their time. Thus, Moore et al. suggested that utilizing an electronic medical record as the first step in screening could be a valuable resource to reduce the burden of work on nurses and help streamline workflow [16].

As our institution introduced this ward-wide electronic sBPA, we sought to troubleshoot the best management of patients in the acute post-operative period in the hope of differentiating normal post-operative physiology from a SIRS response to infection. To increase the specificity of this screening tool, we increased the sBPA triggering threshold in surgical patients. Another study similarly found that the post-operative state made using standardized criteria without modification challenging when assessing for sepsis. Following the National Institute for Health and Care Excellence guideline recommendations, Herrod et al. calculated Quick Sequential Organ Failure Assessment (Q-SOFA) scores on patients at risk for sepsis [17]. They then applied this tool to patients in the 48 hours after colorectal surgery. The researchers looked at the number of patients who triggered Q-SOFA and SIRS criteria and developed infective complications. They found the area under the receiver operating characteristic curve for Q-SOFA to be 0.52 and that for SIRS to be 0.67, indicating that the Q-SOFA was not helpful in this population. In our series, patients having and not having surgery triggered the sBPA at similar rates (33% vs. 35%). However, lactate concentrations that were high enough to change management (>2 mmol/L) were found half as often as in patients not undergoing surgery (13% vs. 25%; p < 0.05). Furthermore, surgical patients had a lactate concentration that prompted transfer to a higher level of care (>4 mmol/L) 1% of the time vs. 3% in the group not having surgery (p < 0.05). Casserly et al. reviewed the Surviving Sepsis Campaign's database; on multivariable analysis, only a lactate concentration >4 mmol/L with or without hypotension was significantly associated with in-hospital death [5]. In accordance with this analysis, our tool was identifying potentially life-threatening sepsis at a third of the rate in patients having versus not having surgery.

After further analysis, a diagnosis of sepsis was made in 5% of surgical patients and 13% of patients not having surgery (p < 0.001) who had a blood sample drawn for lactate measurement after triggering the sBPA. Thus, despite having comparable rates of sBPA firing and blood samples drawn, a diagnosis of sepsis was made significantly less frequently in the surgical group. This phenomenon is confirmed when we compare the sBPA's PPV for the surgical (10%) versus non-surgical (25%) patients. Overall, our PPV was similar to those seen in other studies: Makam et al. performed a systematic review of sepsis alert systems and found that the PPV ranged from 20.5%–53.8% depending on the threshold used to activate the alert [18].

The sBPA had similar sensitivity (64% vs. 67%) and specificity (69% vs. 70%) in detecting sepsis in patients having or not having surgery. The NPV was high in both groups (93% and 97%). This indicates that the sBPA may be useful in excluding patients with an inflammatory response who would not benefit from a sepsis bundle order. However, the low PPV in surgical patients can lead to alert fatigue in nursing and other providers, as well as a suboptimal use of financial and personnel resources. Figure 4 demonstrates that the highest number of normal lactate concentrations was found on post-operative days 1–3. Post-operative day 2 had an average of 140 blood samples drawn, with only 14 of them being abnormal. Nanji et al. reviewed their clinical-decision support alerts in out-patient clinics and ambulatory surgery centers from 2009–2011 [19] and found that 52.6% of the alerts were overridden and of those, only 53% were classified as appropriate overrides. The ideal sBPA would alert providers to potentially sick patients without creating excessive false alarms that blunt the response because of alert fatigue [20]. To avoid alarm fatigue for providers, our protocol does not require the provider to evaluate the patient unless there is both an sBPA and an elevated lactate concentration. This rule also allows providers to identify causes of the elevated lactate concentration and abnormal vital signs that are not the result of sepsis.

Surgical patients who triggered the sBPA were more likely to have had an emergency operation, a wound class of 4, ASA classification >3, and longer operative time than those who did not trigger sBPA. Logically, this is describing patients we would expect to have a more profound inflammatory response after surgery. This is consistent with the findings of MacQueen et al. [21]. General surgery patients undergoing abdomino-pelvic surgery who have a positive screening test based on vital signs were more likely to have cancer (odds ratio [OR] 30.7; 95% confidence interval [CI] 2.2–42.0), an emergency operation (OR 6.5; 95% CI 1.7–2.4), or longer operative time (OR 2.2; CI 1.2–4.1). In addition, patients triggering an sBPA in our study were younger. This may be attributable to the greater physiologic response of younger patients to a systemic insult [22]. Additionally, older patients are more likely to be receiving medications such as beta-blockers that may inhibit their ability to mount a tachycardic response to either the post-operative hyperdynamic state or infection.

The highest incidence of lactate results was seen in the first six days after surgery, most notably in the first two days. But of the patients who had a lactate concentration >4 mmol/L, most of these high values were found between days 7 and 30. Thus, the earlier lactate triggers (particularly those in the first hours after surgery) seem to be more likely to be associated with altered physiology than with sepsis. This view also is consistent with the concept that most post-operative infections take time to mount. For example, SSIs are rare in the immediate post-operative period. When present, they generally are established before the operation or are caused by highly aggressive bacteria resulting in necrotizing soft-tissue infections. Additionally, Moore et al. performed a prospective study of their general surgery patients in the ICU who developed sepsis (5% of patients) [23] and found that 69% of patients who developed sepsis had an abdominal source of infection. In general, infections or abscesses resulting from a perforated hollow viscus, anastomotic leak, or inadequate source control take time to mount, being diagnosed several days to a week after surgery. Thus, this analysis suggests that extending the higher threshold used for surgery patients in the first post-operative day may be useful in increasing the specificity of this tool further. Based on this idea, our hospital extended the higher threshold SIRS criteria trigger to 48 hours post-operatively for incisions categorized as “clean.”

Surgery patients triggering sBPAs had longer hospital stays (9.6 vs. 4.4 days; p < 0.05) and more SSIs (7% vs. 2%, p < 0.05) but a similar mortality rate (3% vs. 4%, p = 0.15) than those who did not trigger an sBPA. Thus, in patients who triggered the sBPA, the additional information gained from a lactate assay did not improve the mortality rate. This result is consistent with the findings of Berger et al., who noted that despite their computerized screening alert increasing both the number of patients with a blood sample drawn and those whose lactate concentration was >4 mmol/L, there was no change in the mortality rate among patients with confirmed sepsis [24]. However, as in our study, MacQueen et al. noted that patients who triggered their vital signs screening test had higher rates of post-operative infection (OR 6.4; CI 1.5–27); therefore, these screening tools may provide a benefit of alerting providers earlier of impending SSIs [21].

There are several limitations our study. It was a single site study; therefore, processes of care unique to our hospital may alter the utilization of this tool at other hospitals. Additionally, our sBPA trigger is based on mSIRs criteria, as these were the recommended critical care criteria at the inception of our study. Also, ICD-9 and CPT codes were used to identify patients with sepsis over the entire hospital course; therefore, it is possible that a specific sBPA event does not indicate sepsis at that time although it could appear later.

Conclusion

Sepsis best practice alerts were triggered at the same rate in patients having and not having surgery, but surgical patients were half as likely to have an abnormal lactate concentration. The surgery patients who triggered the sBPA were more likely to have longer procedures, a wound class of 4, an ASA score of 3 or 4, and emergency procedures and were younger. These findings support the concept that surgery patients are more likely to have altered mSIRS criteria because of non-infectious physiology. However, surgery patients with abnormal lactate concentrations had more SSIs; thus, this tool may speed the time to diagnosis of post-operative infection. At this time, despite the increased blood draws and potential interventions given to this group, no difference was seen in the mortality rate. This automated trigger had a moderate sensitivity and specificity, high NPV, and low PPV for identification of sepsis. The high false-positive rates led to further modification to the trigger for the most common wound class in our non-ICU surgical population. It is unclear whether additional modifications such as incorporating more patient variables in the screening sBPA such as age, co-morbidities, or extending the duration of use of the higher threshold mSIRS criteria an additional 24–48 hours in post-surgical patients could be useful in implementing this tool.