Abstract
This study examines whether Agency for Healthcare Research and Quality Patient Safety Indicators (PSIs) Postoperative Wound Dehiscence (PWD) and Accidental Puncture or Laceration (APL) events reflect problems with hospital processes of care (POC). The authors randomly selected 112 PSI-flagged PWD/APL discharges from 2002-2007 VA administrative data, identified true cases using chart review, and matched cases with controls. This yielded a total of 95 case–control pairs per PSI. Patient information and clinical processes on each case–control pair were abstracted from the electronic medical record (EMR). Although PWD cases and controls differed on incision and closure types, APL cases and controls were comparable in examined processes. Further exploration of the process differences between PWD cases and controls indicated that they were primarily caused by patients’ underlying surgical problems rather than quality of care shortfalls. Documentation of POC was frequently missing in EMRs. Future studies should combine EMR review with alternative approaches, such as direct observation, to better assess POC.
Keywords
The Agency for Healthcare Research and Quality (AHRQ) developed Patient Safety Indicators (PSIs) to screen for potentially preventable patient safety events in the inpatient setting using hospital administrative data. 1 PSIs are commonly used as outcome measures for hospital public reporting (eg, selected PSIs are posted on the Centers for Medicare and Medicaid Services [CMS] Hospital Compare Website). 2 They also are used frequently by health care organizations for their original purpose: quality improvement and case finding. 3 Despite the widespread application of the PSIs, the authors know of only 1 study that has examined whether the occurrence of a PSI event (ie, postoperative venous thromboembolism) is associated with lapses in quality of care such as poor or failed processes of care. 4 According to Donabedian’s framework of quality (ie, structure–process–outcome), 5 examining the relationship between processes and outcomes can provide empirical evidence on whether lapses in processes of care may lead to poor patient outcomes. This information would be helpful for health care providers and quality managers to better identify target areas for quality improvement and allocate resources to modify those processes that will lead to better patient outcomes.
Previous studies showed inconsistent results on the association between processes of care and adverse events.6-10 In particular, conflicting results were found in 2 studies that analyzed complications detected by the precursor of the AHRQ PSIs, the Complications Screening Program (CSP; ie, a computerized method to detect potentially preventable inpatient complications using administrative data). Weingart et al applied physician implicit review and found that cases flagged by the CSP showed more quality problems in clinical processes than unflagged cases, particularly for surgical cases 9 ; another study by Iezzoni et al using explicit chart review did not confirm this association. 10 The authors deduced that the lack of association between CSP events and processes of care may have resulted from the poor validity of the CSP algorithms.
The PSIs were developed to improve on the CSP. Two PSIs, Postoperative Wound Dehiscence (PWD) and Accidental Puncture or Laceration (APL), have been shown to have high positive predictive value in detecting true patient safety events 11 ; both are posted on the CMS Hospital Compare Website for public reporting. 2 Additionally, a study by Rosen et al suggests that these 2 PSIs reflect technical skill–based care. 12 These findings led the authors to explore processes of care in events flagged by these 2 PSIs.
The Veterans Health Administration (VA) is an ideal setting in which to examine whether a PWD/APL event was associated with the quality of inpatient care because of its long-standing emphasis on quality and recent initiatives in patient safety. 13 Additionally, it has a comprehensive data system that enables linkage of centralized electronic medical records (EMRs) to inpatient administrative data. 14 Because of the lack of well-established guidelines on processes of care, the authors selected clinical processes that (a) potentially may be related to PWD/APL based on a literature review15-20 and (b) could be readily extracted from the EMR (see Methods for additional details). The hypothesis was that cases with PWD or APL events were more likely to have shortfalls in processes of care than matched controls without PSI events.
Methods
This matched case–control study used VA administrative and EMR data from October 1, 2002, to September 30, 2007 (fiscal years 2003-2007). Institutional review board approvals were obtained for the study.
Patient Safety Indicator Definitions
PWD was designed to capture cases of fascial dehiscence (ie, reclosure of an unintended separation of the deep tissue layers of the abdominal wall, International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM] code 54.61) following inpatient abdominopelvic surgery. The denominator includes all abdominopelvic surgical discharges in patients 18 years of age or older. To maximize the specificity of the indicator, the denominator excludes cases in which reclosure procedures were performed on the same day as the original surgery, cases for which the length of stay is shorter than 2 days, patients with immunocompromised status, and all obstetric admissions (ie, pregnancy, childbirth). 21
APL was developed to capture the occurrence of an accidental cut, puncture, perforation, or hemorrhage during medical care (defined by ICD-9-CM codes E870.0 through E870.9), or an accidental puncture or laceration during a procedure (ICD-9-CM code 998.2) in a secondary diagnosis field. The denominator includes all surgical and medical discharges in patients 18 years of age or older and excludes patients with APL present on admission or with an APL code in a primary diagnosis field. 21
Data Sources
VA administrative data from the VA’s National Patient Care Database Patient Treatment File (PTF) and EMR data were used. The PTF includes data on patient demographics, principal and secondary diagnoses (ICD-9-CM codes), surgical and nonsurgical procedures (ICD-9-CM codes), and summary information related to each inpatient stay (eg, admission/discharge dates, discharge status). 22 Because the PSIs were designed to screen for safety events occurring in the acute care setting, the nonacute portion of care (eg, rehabilitation, long-term care) was eliminated. 23 The EMR data were obtained via VistAWeb, which allowed centralized access to EMR data from all VA facilities.
Study Population
The authors applied the AHRQ PSI software (version 3.1a) to the PTF file and then selected 28 out of 158 VA acute-care hospitals using a stratified sampling method to obtain a range of facilities in terms of their PSI performance. (Specifically, stratification was based on observed-to-expected PSI cases.) This resulted in a diverse sample of VA hospitals. Details on hospital selection were described previously. 24 For each PSI, 4 software-flagged discharges were randomly selected from each of the 28 hospitals, yielding a total of 112 discharges per PSI.
Case Identification and Processes of Care Data Collection in Cases
Two trained nurses used standardized abstraction instruments, modified from existing AHRQ-developed instruments,25,26 to review medical records of selected discharges for demographics and the occurrence of a safety-related event (PWD or APL). Among the 224 flagged medical records (112 per PSI), a total of 97 true PWD cases and 95 true APL cases were identified. After true cases were identified, nurses continued to abstract patients’ charts for risk factors, clinical process details surrounding the procedure associated with the true event, and patient outcomes.
Because of the lack of well-established guidelines, clinical processes of care that might be associated with the occurrence of PWD/APL were selected based on existing literature. For PWD, data were abstracted on procedure characteristics (eg, name, date, and time when the procedure was performed),27-31 the rank of the physician who performed the procedure (eg, attending vs physician in training),32,33 as well as information on surgical techniques (eg, incision type, suture type and size, closure technique, retention suture placement in operating room).16,19 For APL, data were collected on procedure characteristics (eg, name, time) and physician rank. 20
To better understand patients’ risk factors for PWD, EMR data also were collected on patients’ preoperative conditions (eg, history of chronic obstructive pulmonary disease [COPD], obesity, systemic steroid use)15,18,34 and postoperative outcomes (eg, development of postoperative wound infection, pneumonia) 34 that may increase the risk of PWD. No specific patient risk factors were abstracted from the EMR for APL as there was little information from the literature regarding which risk factors were associated with APL.
Interrater Reliability
Interrater reliability (IRR) was assessed twice (ie, at the beginning and toward the end of the abstraction process) for both PSIs. Specific to PWD, IRR testing revealed 95% agreement (N = 48 questions) on the initial 5 records and 96% on subsequent testing. 25 For APL, the initial IRR achieved 97% (N = 29 questions) and the subsequent IRR was slightly lower at 94%. 26 The IRR assessment process is described in detail elsewhere.24-26
Control Matching and Processes of Care Data Collection in Controls
True cases (97 PWD and 95 APL) were matched to controls based on age, sex, hospital, and diagnosis-related group (DRG; which accounts for procedure type). For cases that had multiple matched controls defined by these variables, the control with the closest probability of developing a PWD or APL event was chosen. The probabilities were estimated using a logistic regression model that included age, sex, DRG groups, and comorbidities (as defined in the AHRQ comorbidity software) from the index admission as independent variables. 35 A matched control was found for all but 2 PWD cases, yielding a total of 95 case–control pairs per PSI.
The same nurses used a shortened version of the abstraction tool to review the EMR of the 190 controls (95 per PSI) to collect information on demographics, risk factors, and processes of care. Similar process details surrounding the procedures that occurred during hospitalization were abstracted for controls, as they had been for cases. Rather than collecting information on the specific procedure associated with the PSI event (as was done with cases), information was collected on all relevant procedures performed on the controls because they did not experience any of the selected PSIs.
Comorbidity Data From the Patient Treatment File
To obtain a more comprehensive picture of patients’ comorbidities beyond that of the EMR, chart-reviewed data were supplemented with comorbidity information from the PTF file. The AHRQ Comorbidity Software v.3.1 35 was applied to the administrative data to flag comorbidities based on patients’ diagnostic codes.
Analysis
Cases and controls were compared on a number of variables including demographics, patient risk factors, procedure characteristics, and other collected processes of care. t Tests for were performed for continuous variables and χ2 tests for categorical variables. All statistical analyses were performed using SAS version 9.1 (SAS Institute, Inc, Cary, NC).
To compare procedure-related factors, the procedure associated with the patient safety event was used in all cases. For PWD controls, the first abdominopelvic procedure was used; for APL controls, all procedures occurring among controls were used because, theoretically, APL can occur in any procedure. Given the inherent differences between operating room (OR) and non-OR procedures (ie, OR procedures usually are more complex and involve a larger team than non-OR procedures), APL procedures were further categorized into these 2 groups and analyzed separately.
The Clinical Classifications Software (CCS) was applied, based on ICD-9-CM procedure codes, to further categorize surgical procedures into more homogeneous groupings. 36 In addition to DRGs, cases and controls for each PSI were matched based on the CCS procedure groups, yielding a subset of 33 case–control pairs for PWD and 73 case–control pairs for APL. Sensitivity analyses were conducted to compare patient characteristics and clinical processes among these 2 matched subsamples and the original matched samples.
Results
Population Characteristics and Risk Factors
Cases and controls for both PSIs were similar with respect to age and sex. However, with regard to race, both PWD and APL cases were less likely to be white than controls. Mean length of stay of PWD cases was longer than that of controls, but similar in both APL cases and controls (Table 1).
Population Characteristics.
Abbreviations: PWD, postoperative wound dehiscence; APL, accidental puncture or laceration; SD, standard deviation; LOS, length of stay; BMI, body mass index.
P < .01.
BMI is calculated by patient’s height and weight abstracted from the electronic medical record.
The comorbidities obtained from the PTF did not vary between cases and controls for either PSI. However, chart-abstracted data showed that PWD cases had a higher prevalence of a history of COPD and obesity and experienced significantly more postoperative wound infections and pneumonia than controls (Table 1).
The sensitivity analyses also showed similar results. For example, among 33 PWD case–control pairs that were further matched on procedures, cases had longer length of stay (30 days vs 11 days, P < .01), greater COPD (21% vs 3%, P < .01) and obesity (27% vs 12%, P < .01), and a higher incidence of postoperative wound infections and pneumonia than controls (21% vs 6%; 9% vs 0%, P < .01). For APL, no significant differences were found in the examined characteristics among 73 pairs of cases and controls (Table 1).
Postoperative Would Dehiscence
In general, PWD cases and controls underwent comparable procedures (ie, most of the patients had gastrointestinal surgeries). However, significant differences in surgical techniques (ie, incision type, closure techniques) were found between cases and controls. Cases more frequently had a “midline vertical” incision (an incision that extends vertically along the length of the abdomen that may lead to a higher risk of PWD) 19 than controls, whereas controls had a variety of other types of incisions. Study physicians (MC, AB) reviewed the 12 cases with midline incisions in more detail; they concluded that for all cases, performing a midline incision was appropriate based on the patient’s clinical condition.
Although slight differences were found in other examined processes of care (eg, cases underwent more procedures on the weekend than controls), none of these was statistically significant. Furthermore, missing documentation in the EMR varied across examined processes of care (eg, 10% of cases and 13% of controls were missing documentation of incision type; Table 2). Similar results were found in the sensitivity analyses among the 33 PWD case–control pairs matched further on CCS procedure groups.
Postoperative Wound Dehiscence (PWD)–Specific Procedure and Physician Characteristics. a
Information also was collected on whether the index procedure involved a repair of an existing defect/hole in the abdominal wall fascia/skin, and suture type size, but none of these were significantly different between PWD cases and controls.
Accidental Puncture or Laceration
The total number of OR and non-OR procedures performed for cases and controls is shown in Table 3. The procedure categories were comparable between cases and controls for both OR and non-OR procedures. Furthermore, there were no significant differences in procedure day/time or in physician rank between cases and controls with respect to either type of procedure. Documentation of physician rank was missing for a total of 7 procedures in cases and 4 procedures in controls (Table 3). All results were similar in the sensitivity analyses among the 73 pairs of APL cases and controls that were further matched on CCS procedure groups.
Accidental Puncture or Laceration–Specific Procedure and Physician Characteristics.
Abbreviation: OR, operating room.
Examples of non-OR procedures: small bowel endoscopy, cardiac catheterization, hemodialysis, and insertion of endotracheal tube.
Discussion
The authors were unable to confirm the association between examined processes of care and PWD/APL events in this study. Although there were differences in incision types and closure techniques between PWD cases and controls, these did not indicate problems related to quality of care. For APL, cases and controls were comparable in examined procedure and physician characteristics using both types of matching approaches.
Previous clinical evidence suggests that “midline vertical” incision has a higher risk of PWD than other types of incisions and should not be used if alternative options are available. 19 PWD cases and controls were found to have experienced comparable procedures, but cases had more “midline vertical” incisions than controls. However, based on physician review, these differences were clinical in nature; none was found to be an inappropriate incision or an indication of poor quality of care. Similarly, the different fascial closure types between PWD cases and controls also do not indicate process shortfalls.
With respect to APL, the results suggest that this event occurred “randomly” across facilities and practices. Because a previous study found that APL events were most commonly associated with gastrointestinal, cardiothoracic, and orthopedic procedures in pediatric surgeries, 20 DRG was included as one of the matching criteria in this study to account for the type of procedure a patient had. As expected, this matching successfully yielded comparable OR and non-OR procedures between APL cases and controls, and no differences in either physician expertise or the day/time of procedures between APL cases and controls. The sample was further matched based on CCS procedure groups for the sensitivity analyses. Although other studies have demonstrated positive effects of physician expertise on reducing complications32,33 and mixed results in determining whether nighttime procedures lead to higher risk of complications27-31 (ie, some studies confirmed the higher risk of postoperative complications for nighttime procedures while others did not), these associations could not be confirmed in the occurrence of either APL or PWD.
Although the negative findings in both this study and the previous work by Iezzoni et al 10 suggest that these PSIs/complications were not associated with examined processes of care, researchers should not overlook some of the limitations in using chart review to examine clinical processes, such as incomplete chart documentation of process details 37 and discrepancies between chart documentation and what actually occurred in the medical encounter.38-40 In this study, it was found that about a quarter of the PWD cases and controls were missing documentation of the surgical technique used to close fascia, and a quarter of the cases and half of the controls were missing the surgical technique used to close the skin (Table 2). It also was noted that medical charts did not document some surgical details that may be associated with the occurrence of PWD events (eg, stitch length). These shortcomings suggest supplementing chart review with other approaches (eg, procedure-level direct observations) to examine processes of care in future studies.
In addition to the limitations of using chart review data to examine clinical processes, this study had 2 other limitations. First, the sample size (ie, 95 case–control pairs for each PSI) was moderate, because a convenience sample of true cases that was identified in the authors’ previous studies25,26 was used and then matched with controls. Second, cases and controls were matched based on the administrative data that only contained information about the index admission (ie, the comorbidities flagged in administrative data must have been coded during the index admission). Therefore, the matching may not have accounted for all of a patient’s conditions (eg, history of COPD, obesity) that may increase the risk of experiencing a PSI event. Additionally, the severity of disease is not available from administrative data. Finally, because matching on DRGs does not yield a perfect match on specific surgical procedures, sensitivity analyses were conducted using a subset of the sample that matched cases and controls on CCS procedure groups to improve matching by procedure type. However, these results did not change based on more closely matching surgery type.
Despite the limitations noted, this study had several strengths. This is the first attempt to explicitly explore the clinical processes that might be associated with the occurrence of PWD/APL events. Future studies can build on this study to further examine the relationship between processes of care and PSI events. Second, the authors were able to identify true events using a gold standard (ie, chart review) before matching the controls. This allowed for a more valid sample of cases (ones that experienced a PSI event during the hospitalization) than previous studies that used only cases flagged by a computerized algorithm (ie, false positives likely would have been included).9,10 Third, matching based on age, sex, hospital, DRG (or CCS grouping), and comorbidities enhanced the ability to explore the relationship between processes of care and PWD/APL events in a comparable sample of cases and controls.
Conclusion
The occurrence of PWD or APL was not associated with poor quality of care based on chart review. Future research should continue to explore other clinical processes that may be associated with these events. To capture a more complete and accurate picture of what occurred clinically, future studies may consider using alternative approaches such as direct observation.
Footnotes
Acknowledgements
Special thanks to Kathleen Hickson, PhD, RN, Sally MacDonald, RN, Julie Lynch, PhD, RN, and Kristine Ruggerio, RN, for their assistance with the abstraction instrument and guidelines modification, data abstraction, and data cleaning; Amresh Hanchate, PhD, for his advice on methodology; Susan Loveland, MAT, for programming expertise and assistance with identifying the hospital sample and flagged cases; and Enzo Yaksic, BS for the literature review.
Declaration of Conflicting Interests
The authors declared no conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the Department of Veterans Affairs Health Services Research and Development Service (Grant Number SDR 07-002).