Rethinking patient safety: Why is patient safety event identification so challenging in day-to-day operations?

Despite the significant prioritization of safety work since the Institute of Medicine's 1999 To Err is Human report, ¹ there is evidence to suggest that improvements have been minimal.^2,3 The basis for effective improvement is contingent on a framework to accurately capture current rates of harm. There has been much attention given to the best safety measurement approach, which encompasses what is collected, how it is collected, why it is collected, and what it means. Voluntary and mandatory incident reporting, trigger tools, retrospective review, and use of reimbursement codes have emerged as common approaches to safety measurement. Despite the number of approaches that have been used both independently and in combination, it has been difficult to prescribe the best approach and most importantly, how to determine which healthcare organizations are ‘more’ or ‘less' safe than others.

Comprehensive estimates of patient harm are difficult, especially outside of a research protocol, and discrepancies exist with different measurement systems. For example, estimates suggest that adverse event rates in pediatric inpatients have not improved from 2007 to 2012, with pediatric adverse event rates substantially higher in academic teaching hospitals and in patients with chronic comorbidities. ² Other estimates corroborate the hypothesis that preventable harm persists and has worsened post-COVID.^4,5

Despite the widespread healthcare safety improvement efforts over the last two decades, inadequate measurement processes have made it difficult to truly understand their impact. Estimates of the rates of patient harm are often very different than those experienced at the hospital level without an ongoing harm identification research protocol. Furthermore, there is likely significant harm that is not being captured by our current measurement processes. The inadequacy of existing measurement systems has been the topic of conversation for many experts in the field. For example, Aaronson and Bates suggest that there are three major drivers to the measurement challenge: inability to distinguish between changes in harm rates and changes in documentation, variability in outcomes based on the approach to measurement used, and the measurement challenge does not impact each adverse event type equally (i.e., adverse drug events versus diagnostic errors). ⁶

Others have highlighted the need to shift away from measurement of harm to measurement of safety through normal work to truly diagnose an organization's safety capability irrespective of the outcome. ⁷ All of these measurement challenges, in combination, result in a variety of literature highlighting discrepant rates of harm depending on the measurement approach used.^8,9

Because what we measure is what we perceive, and what we perceive is what we prioritize, our time, money, and resources are, understandably, being funneled toward safety metrics that are high priority because they are visible, not necessarily because they are highest priority due to assessments of most common or highest harm. The purpose of this commentary is to highlight three measurement deficiencies that have been largely overlooked but contribute significantly to the measurement challenge. In doing so, we aim to deepen our understanding of the many challenges we face as a safety community, enabling us to make more informed decisions about measurement going forward.

Challenge 1: what you see is what you prioritize

Voluntarily reported incidents continue to significantly inform safety priorities in healthcare, despite their well-known limitations, such as but not limited to the need for protection against disciplinary action, ease of reporting, and inadequate confidentiality or anonymity.^10–12 In this section, we will specifically focus on the role voluntary reporting plays in informing resourcing decisions. Voluntary reporting is a skewed view of the entire scope of events and given the fact that most hospital incident reporting systems do not capture most patient harm, ¹³ we will continue to pursue efforts based on a misleading precedent. To illustrate, adverse drug events (ADEs) present the single greatest risk of harm to patients in hospitals.^2,4 Traditional efforts to detect ADEs have focused on voluntary reporting and tracking of errors identified. However, public health researchers have established that only 10 to 20% of errors are ever reported, and, of those, 90 to 95% cause no harm to patients. ¹⁴ A multicenter comparison of medication errors done by audit revealed only 1.1% of all serious prescribing errors were reported to the hospitals’ voluntary reporting system and only 3 in every 1000 clinical prescribing errors identified at audit were reported. ¹⁵

What gets reported is a derivative of what is framed as important to report, which is influenced by what the reporter perceives as important to the organization: If the organization places emphasis on falls, falls are more likely to be reported due to the higher awareness, knowledge of what constitutes a fall, and attention after a patient falls. Therefore, what the organization ‘sees’ as its safety risks is limited and what is defined as real is real in its consequences.

While most healthcare organizations do not rely solely on voluntary incident reporting, it is still difficult to get a full picture of an organization's safety status. Currently, organizations commonly synthesize safety data from several sources: claims, litigation results, administrative data, infection control measures, voluntary event reporting, and occasionally from trigger tools. Yet, each safety event detection system provides differing results, largely given the differences in terminology (see Challenge 3). ¹⁶ Until safety events are comprehensively captured and triangulated in a meaningful and consistent way, a hospital will not know how safe their performance is when compared to months or years past and will be unable to make prioritization decisions based on the most impactful safety event types.

Challenge 2: bias in what is seen

While it is recognized that organizations need to incorporate safety data from multiple sources to optimally capture safety events and vulnerabilities, this may still not be enough to capture the totality of safety events, particularly for different groups of people. ¹⁷ Emerging evidence suggests that there are disparities not only in data capture but also in the rates of experienced safety events. The pediatric collaborative, Solutions for Patient Safety PHARE cohort data suggest that patients in multiracial Hispanic and combined Hispanic categories suffered higher rates of central line associated blood infections and that Black or African American patients experienced a significantly higher rate of unplanned extubations. This data aligns with similar findings within the adult population. ¹⁸ Again, because what we see is what we act on, it is essential to inquire about the disparities in the measurement approach itself. For example, research has emerged to suggest that implicit bias may lead to inadequate capture of safety events that occur involving vulnerable communities, such as Latino patients or patients with Medicaid. ¹⁹ These results are further supported by studies that have examined the racial disparities in rates of harm based on the mechanism for data capture used. ²⁰

Challenge 3: confusion in what we are seeing

Even within the harm rates that we do see, definitional confusion can stunt our ability to do anything meaningful with it. The terms ‘safety event,’ ‘adverse event,’ ‘harm event,’ ‘preventable safety event,’ ‘sentinel event,’ and ‘serious safety event’ provide only a smattering of terms that comprise safety vernacular. These terms have yet to be reconciled among leading safety bodies. The World Health Organization upholds ‘adverse event’ in their language, defining it as ‘an incident which resulted in harm to a patient.’ ²¹ The US Department of Health and Human Services’ Office of the Inspector General, also endorses ‘adverse event’ but defines it as ‘an event in which care resulted in an undesirable clinical outcome, an outcome not caused by an underlying disease, that prolonged the patient stay, caused permanent patient harm, required life-saving intervention, or contributed to death.’ ²² ‘Sentinel event’ a term largely popularized in the US by the accrediting body, The Joint Commission, is defined as ‘a patient safety event that results in death, permanent harm or severe temporary harm.’ ²³ Already, we can see that the same term, defined differently, can influence different conclusions about the state of patient safety in an institution.

Given the challenges above, comparing rates of patient harm between institutions becomes nearly impossible. The definitional ambiguity may depict a false interpretation of an organization's performance against any established benchmarks. Patient safety research that documents harm events or adverse events are based on non-harmonized language, institutions adhere to their chosen definitions, and the lack of standardization makes comparison challenging. ²⁴ Methodological differences hinder our ability to compare organizational performance and study results. Take adverse drug events again as an example. A study in pediatrics revealed the incidence for adverse drug events can span from 0 to 43 per 100 admissions. Depending on the study, there are zero ADEs during a study period or as many as 40% of all admissions have an ADE. The difference is not likely rooted in patient severity of illness, complexity, or hospital safety standards but more likely, this reveals the variability of measurement methods, definition application, and reporting standards within pediatrics.^25,26