Comparison of the Agency for Healthcare Research and Quality Patient Safety Indicator Rates Among Veteran Dual Users

Abstract

This study compares rates of 11 Agency for Healthcare Research and Quality Patient Safety Indicators (PSIs) among 266 203 veteran dual users (ie, those with hospitalizations in both the Veterans Health Administration [VA] and the private sector through Medicare fee-for-service coverage) during 2002 to 2007. PSI risk-adjusted rates were calculated using the PSI software (version 3.1a). Rates of pressure ulcer, central venous catheter-related bloodstream infections, and postoperative sepsis, areas in which the VA has focused quality improvement efforts, were found to be significantly lower in the VA than in the private sector. VA had significantly higher rates for 7 of the remaining 8 PSIs, although the rates of only 2 PSIs (postoperative hemorrhage/hematoma and accidental puncture or laceration) remained higher in the VA after sensitivity analyses were conducted. A better understanding of system-level differences in coding practices and patient severity, poorly documented in administrative data, is needed before conclusions about differences in quality can be drawn.

Keywords

patient safety indicators quality measurements patient safety quality of care coding variation

Since the Institute of Medicine released the report To Err Is Human,¹ patient safety has become a national focus in both the Veterans Health Administration (VA) and the private sector. Patient Safety Indicators (PSIs) are a set of standardized, evidence-based measures of patient safety developed by the Agency for Healthcare Research and Quality (AHRQ) to flag potentially preventable inpatient adverse events using administrative data.² Studies have shown that the PSIs have moderate to high criterion validity in both the VA and the private sector; the positive predictive values (ie, the percentage of true events out of all PSI-flagged cases) are generally comparable between the 2 settings.^3-8 Selected PSIs are currently used for public reporting by the Centers for Medicare and Medicaid Services (CMS).⁹

Previous studies have compared PSI rates between the VA and the private sector; however, results are generally inconsistent. For example, some PSIs, such as postoperative physiologic and metabolic derangements and infection due to medical care, were found to have higher rates in the private sector,^10,11 whereas another study showed the opposite.¹² These discrepancies may be due largely to differences in case mix; of note, the AHRQ PSI risk-adjustment software was developed based on the Healthcare Cost and Utilization Project (HCUP) population, which has a different case mix than the population of veterans who use the VA as their primary source of care.^10-12 To address this issue, Weeks et al suggested limiting such comparisons to veteran dual users who concurrently obtain care both within and outside the VA because this method would compare the same group of patients in both settings.¹¹

This study compares the rates of 11 previously validated PSIs in a nationwide sample of veteran dual users. The null hypothesis is that dual users will have similar PSI rates and likelihood of developing a PSI event when hospitalized in the VA compared to the private sector. In addition, the factors that may influence these comparisons of PSI rates are explored, such as variations in coding across settings and differences in the reasons why veteran dual users seek care in the VA versus the private sector.

Methods

This was a retrospective cross-sectional study using VA administrative data and Medicare fee-for-service claims data for VA fiscal years (FYs) 2003 to 2007 (October 1, 2002, to September 30, 2007). Institutional review board approvals were obtained from the VA Boston Healthcare System.

Data Sources

A total of 1 057 574 patients with 2 343 088 VA acute care hospitalizations (VA hospitalizations) were identified from the VA’s National Patient Care Database Patient Treatment File (PTF). The PTF consists of patient demographics and information on hospitalizations including admission/discharge dates, flags for acute versus nonacute care, discharge status, diagnoses, and surgical and nonsurgical procedures.¹³ These data were merged with the Medicare Provider Analysis Review (MedPAR) file, using veterans’ encrypted social security numbers, to identify all hospitalizations in the private sector through Medicare fee-for-service coverage (“Medicare Hospitalizations”; n = 785 529). The patient characteristics and information on hospitalizations in the MedPAR file are similar to the information available in the PTF.¹³

Patient Sample and Hospitalizations

Figure 1 presents the final patient sample used in this study. Of the 1 057 574 veterans who had acute hospitalizations in the VA during FYs 2003 to 2007, approximately 75% were VA-only users (ie, veterans having only VA hospitalizations over the study period); the balance were dual users (ie, veterans having at least 1 VA and 1 Medicare hospitalization). VA-only users had more than 1.6 million VA hospitalizations, compared to 1 462 077 hospitalizations (676 548 VA hospitalizations and 785 529 Medicare hospitalizations) among dual users during FYs 2003 to 2007.

Figure 1.

Patient population and hospitalizations.

For the study analyses (comparing patient safety between the VA and the private sector), dual users younger than age 65 in FY 2003 (n = 69 387) were excluded because their eligibility for Medicare is based on disability or end-stage renal disease rather than age; furthermore, their comorbidities and patterns of health care utilization are often different from for those age 65 and older.¹⁴ The final sample of dual users along with the breakdown of VA versus Medicare hospitalizations is shown in Figure 1.

PSIs

Eleven of 18 hospital-level PSIs were selected that have been validated in previous studies^2-6,8: PSI 3, pressure ulcer (ulcer); PSI 6, iatrogenic pneumothorax (pneumothorax); PSI 7, central venous catheter-related bloodstream infections (CR-BSIs); PSI 8, postoperative hip fracture (fracture); PSI 9, postoperative hemorrhage or hematoma (hemorrhage/hematoma); PSI 10, postoperative physiologic and metabolic derangement (derangement); PSI 11, postoperative respiratory failure (respiratory failure); PSI 12, postoperative pulmonary embolism or deep vein thrombosis (PE/DVT); PSI 13, postoperative sepsis (sepsis); PSI 14, postoperative wound dehiscence (dehiscence); and PSI 15, accidental puncture or laceration (puncture/laceration).¹⁵ (Complete definitions of these PSIs are available from the authors on request.) The positive predictive values of 6 PSIs (pneumothorax, hemorrhage/hematoma, derangement, respiratory failure, dehiscence, and puncture/laceration) were greater than 60%, suggesting moderate to strong validity.³ Some of these PSIs also have been endorsed by the National Quality Forum.¹⁶

Application of PSI Algorithms

The AHRQ PSI software (version 3.1a) was applied to the merged VA–Medicare data set of dual users. Three surgical PSIs (ie, derangement, respiratory failure, and sepsis) required information on whether a hospitalization was elective or urgent/emergent.¹⁵ Because VA data lack hospitalization type, a previously developed algorithm was used to impute “elective” flags for both VA and Medicare data in order to maintain consistency between the 2 settings.¹⁷

For each PSI, the software generated the observed, expected, and risk-adjusted rate with 95% confidence intervals (CIs). The observed rate was the number of PSI-flagged cases (numerator) divided by the total number of PSI-eligible cases (denominator). The expected rate was estimated based on the case mix (age, sex, and the presence of 27 comorbidities) of PSI-eligible hospitalizations. AHRQ comorbidity software was used to obtain specific comorbidities based on secondary diagnosis codes from the index hospitalization.¹⁸ The risk-adjusted rate was calculated as the observed-to-expected ratio multiplied by the reference population (ie, the population rate in the HCUP State Inpatient Database).¹⁵

Analyses

First, the research team compared the characteristics of hospitalizations (ie, patient’s age, sex, length of stay [LOS] in days, and comorbidities) between VA-only users and dual users in order to examine differences in sociodemographics and health status. The team then compared these characteristics between VA and Medicare hospitalizations among dual users. Statistical tests were run (t tests for continuous characteristics and χ² tests for categorical characteristics) to determine whether the differences in hospital characteristics were significant.

As a next step, the research team compared the risk-adjusted PSI rates between VA and Medicare hospitalizations among dual users and examined whether the 95% CIs overlapped (ie, the approach that CMS uses to determine performance outliers for public reporting).⁹ The team then developed separate logistic regression models to predict the likelihood of experiencing each individual PSI event and “any PSI event” (a dichotomous outcome, yes/no), adjusting for the risk factors included in AHRQ comorbidity software as well as type of hospitalization (VA or Medicare). The odds ratio of developing a PSI event in the VA was calculated using Medicare hospitalizations as the reference group.

Sensitivity Analyses

Several sensitivity analyses were conducted to investigate the factors that may influence comparisons of the PSI rates. First, the literature suggests that coding practices may vary across settings. In the private sector, International Classification of Diseases, Ninth Revision, Clinical Modification codes listed in administrative data are directly related to payments to hospitals. As a result, coders in the private sector have incentives to “upcode” patient severity in order to increase reimbursements.¹⁹ These incentives do not exist in the VA. To address concerns about the impact of potential private sector upcoding, the risk-adjusted PSI rates for Medicare hospitalizations were recalculated based on the expected rates generated from VA hospitalizations. This reduces the impact of coding differences when comparing dual user risk-adjusted PSI rates by source of care (VA or Medicare).

Second, both settings may be susceptible to incomplete coding of adverse events.^20,21 VA administrative data allow an unlimited number of slots for diagnosis and procedure codes, whereas only a total of 10 diagnoses and 6 procedure slots were available in the Medicare claims data. Adverse events associated with no/minor injuries may be less likely to be coded in the Medicare data because of limited coding slots. Thus, risk-adjusted PSI rates in the VA were recalculated based on only the first 10 diagnoses and 6 procedures in VA administrative data in order to more equitably control for similar numbers of diagnoses/procedures in both settings.

Third, the medical reason for hospitalization may vary symmetrically between the VA and the private sector (eg, veterans with emergent medical conditions are likely to be sent to the nearest private hospital rather than a VA hospital).^14,22-24 Thus, the regression models were reestimated for a subgroup of VA and Medicare hospitalizations that was matched by diagnosis-related groups (DRGs) in order to more accurately compare dual users who had similar reasons for hospitalization. Because the DRGs represent broad categories and may not be specific enough to yield similar comparisons of medical conditions, these models also were reestimated by controlling for the principal diagnosis of the hospitalization.

Furthermore, the models were run by including only the first hospitalization (either VA or Medicare hospitalization) after a “180-day clean period” (ie, the patient was not hospitalized for 180 days prior to this admission) to minimize potential bias from patients who are admitted frequently.

Last, dual users who obtained care disproportionately across the 2 settings were excluded to control for any dissimilar patterns of utilization in either setting. Specifically, 3 rounds of analyses were conducted, eliminating veterans with 10, 4, and 2 times as many VA hospitalizations as Medicare hospitalizations, and vice versa.

Results

Table 1 shows that dual users were older and had a higher mean number of comorbidities than VA-only users. As shown in Table 2, mean LOS among dual users was longer in the VA than in the private sector. The number of comorbidities was comparable between VA and Medicare hospitalizations; however, dual users hospitalized in the private sector had a higher prevalence of congestive heart failure, chronic lung disease, and renal failure than those hospitalized in the VA.

Table 1.

Comparisons of Hospitalization Characteristics Between VA-Only Users Hospitalized in the VA and Veteran Dual Users Hospitalized in the VA and the Private Sector Through Medicare Fee-for-Service Coverage^a.

	VA Hospitalizations Among VA-Only Users	VA and Medicare Hospitalizations Among Dual Users
Number of admissions	1 666 540	1 027 053
Age, mean (SD)	63.0 (12.8)	70.6 (11.6)
Length of stay (days), mean (SD)	6.6 (8.6)	6.8 (11.1)
Male, %	96.2	97.5
Number of comorbidities, mean (SD)	1.7 (1.3)	2.2 (1.3)
Comorbidities, %
Congestive heart failure	6.0	13.6
Chronic lung disease	16.7	26.8
Diabetes	26.0	32.4
Renal failure	5.4	11.9
Liver disease	4.7	3.2
HIV/AIDS	0.5	0.4
Obesity	4.0	3.1
Alcohol/drug abuse	9.5	7.9
Psychoses	4.4	5.0
Depression	6.4	6.0
Hypertension	49.2	52.8

Abbreviation: VA, Veterans Administration.

All comparisons yielded P < .0001.

Table 2.

Characteristics of VA Versus Private Sector (Medicare) Hospitalizations Among Veteran Dual Users^a.

	VA Hospitalizations	Medicare Hospitalizations
N	473 016	554 037
Age, mean (SD)	76.5 (6.4)	76.8 (6.8)
Male, %	98.1	98.2
Length of stay (days), mean (SD)	7.3 (8.2)	6.5 (12.5)
Comorbidities, mean (SD)	2.1 (1.4)	2.2 (1.3)
Selected comorbidities, %
Congestive heart failure	12.5	17.8
Chronic lung disease	24.3	32.3
Diabetes	33.1	31.2
Renal failure	9.8	13.2
Liver disease	1.5	1.5
HIV/AIDS	0.1	0.1
Obesity	2.3	2.2
Alcohol/drug abuse	2.8	4.4
Psychoses	3.0	2.9
Depression	4.9	4.8
Hypertension	56.3	53.0

Abbreviation: VA, Veterans Administration.

All comparisons yielded P < .0001.

Table 3 displays comparisons of PSI observed, expected, and risk-adjusted rates between VA and Medicare hospitalizations. The observed rates varied from 0.6 per 1000 hospitalizations at risk for pneumothorax to 25.9 per 1000 for ulcer in the VA, compared to a range from 0.6 per 1000 for pneumothorax to 44.4 for ulcer among Medicare hospitalizations. The expected rates were lower in the VA for all PSIs except hemorrhage/hematoma. With regard to the risk-adjusted rates, dual users had significantly higher rates of 3 PSIs (ulcer, CR-BSIs, and sepsis) when hospitalized in the private sector than in the VA. However, dual users had significantly lower rates in the private sector than in the VA for 7 PSIs (pneumothorax, fracture, hemorrhage/hematoma, derangement, respiratory failure, dehiscence, and puncture/laceration per 1000 hospitalizations at risk); all 95% CIs did not overlap.

Table 3.

The Observed Rates (O), Expected Rates (E), and Risk-Adjusted Rates (RA) for Selected PSIs With 95% Confidence Intervals per 1000 Population at Risk for Veteran Dual Users Hospitalized in the VA Versus in the Private Sector Through Medicare Coverage.

	VA Hospitalizations (n = 473 016)				Medicare Hospitalizations (n = 554 037)				Medicare Hospitalizations With Replacement of VA Expected Rates (n = 554 037)
PSIs^a	O	E	RA	95% CI of RA	O	E	RA	95% CI of RA	O	E	RA	95% CI of RA
Ulcer	25.9	29.5	20.4	(19.9, 21.0)	44.4	39.1	27.8^b	(27.3, 28.3)	44.4	29.5	36.9^b	(36.3, 37.6)
Pneumothorax	0.6	0.4	0.8	(0.7, 0.9)	0.6	0.6	0.5^b	(0.5, 0.6)	0.6	0.4	0.8	(0.7, 0.8)
CR-BSIs	2.0	2.0	2.1	(2.0, 2.3)	3.5	2.8	2.7^b	(2.6, 2.9)	3.5	2.0	3.7^b	(3.5, 3.9)
Fracture	0.8	0.3	0.6	(0.5, 0.7)	0.7	0.5	0.4^b	(0.3, 0.5)	0.7	0.3	0.6	(0.4, 0.7)
Hemorrhage/hematoma	4.7	3.9	3.3	(3.0, 3.6)	2.8	3.7	2.1^b	(1.9, 2.4)	2.8	3.9	2.0^b	(1.7, 2.3)
Derangement	3.3	6.8	0.5	(0.4, 0.7)	1.3	9.4	0.2^b	(0.1, 0.3)	1.3	6.8	0.2^b	(0.1, 0.3)
Respiratory failure	23.4	16.8	12.7	(11.9, 13.6)	30.2	27.9	9.9^b	(9.4, 10.5)	30.2	16.8	16.5^b	(15.6, 17.4)
PE/DVT	15.7	12.9	11.9	(11.3, 12.5)	16.1	14.3	11.0	(10.5, 11.5)	16.1	12.9	12.2	(11.6, 12.7)
Sepsis	10.6	12.6	9.2	(7.6, 10.9)	29.2	19.6	16.3^b	(15.2, 17.5)	29.2	12.6	25.5^b	(23.6, 27.3)
Dehiscence	9.4	4.4	4.4	(3.8, 4.9)	5.3	5.9	1.8^b	(1.3, 2.3)	5.3	4.4	2.4^b	(1.8, 3.1)
Puncture/laceration	2.5	1.6	5.6	(5.3, 5.8)	1.9	1.7	3.9^b	(3.7, 4.1)	1.9	1.6	4.2^b	(4.0, 4.5)

Abbreviations: CI, confidence interval; PSI, Patient Safety Indicator; VA, Veterans Administration.

Abbreviations of PSIs: Ulcer—PSI 3, pressure ulcer; Pneumothorax—PSI 6, iatrogenic pneumothorax; CR-BSIs—PSI 7, central venous catheter-related blood stream infections; Fracture—PSI 8, postoperative hip fracture; Hemorrhage/hematoma—PSI 9, postoperative hemorrhage or hematoma; Derangement—PSI 10, postoperative physiologic and metabolic derangement; Respiratory failure—PSI 11, postoperative respiratory failure; PE/DVT—PSI 12, postoperative pulmonary embolism/deep vein thrombosis; Sepsis—PSI 13, postoperative sepsis; Dehiscence—PSI 14, postoperative wound dehiscence; Puncture/laceration—PSI 15, accidental puncture or laceration.

95% CI of risk-adjusted PSI rate among Medicare hospitalizations did not overlap with that among VA hospitalizations.

Table 4 shows that the odds of developing any one of the examined PSI events was lower in the VA than the private sector. However, there were differences when examining individual PSIs. Similar to the differences in risk-adjusted PSI rates, the risk-adjusted odds among dual users of developing ulcer, CR-BSIs, respiratory failure, and sepsis were significantly lower when dual users were hospitalized in the VA. However, the odds of experiencing 4 PSIs—hemorrhage/hematoma, derangement, dehiscence, and puncture/laceration—were significantly higher in the VA than the private sector. No significant differences were found in odds of experiencing pneumothorax, fracture, or PE/DVT across settings.

Table 4.

The Odds Ratio (OR) of Experiencing a PSI Event for Veteran Dual Users Hospitalized in the VA Versus in the Private Sector Through Medicare Coverage.

PSIs^a	Eligible Hospitalizations for Regression	OR (95% CI)
Ulcer	425 333	0.65 (0.63, 0.68)^b
Pneumothorax	963 505	1.14 (0.96, 1.35)
CR-BSIs	684 742	0.62 (0.56, 0.68)^b
Fracture	131 345	1.20 (0.80, 1.81)
Hemorrhage/hematoma	177 641	1.73 (1.48, 2.03)^b
Derangement	86 790	3.21 (2.34, 4.39)^b
Respiratory failure	59 475	0.87 (0.78, 0.98)^b
PE/DVT	175 337	1.04 (0.96, 1.13)
Sepsis	27 506	0.37 (0.30, 0.45)^b
Dehiscence	23 371	2.23 (1.60, 3.10)^b
Puncture/laceration	1 022 775	1.30 (1.20, 1.42)^b
Any PSI^c	1 026 977	0.88 (0.85, 0.90)^b

Abbreviations: PSI, Patient Safety Indicator; VA, Veterans Administration.

A total of 853 hospitalizations (0.008%) were flagged by multiple PSIs.

P < .0001.

The sensitivity analyses changed some of the results. When risk-adjusted PSI rates for Medicare hospitalizations were recalculated using VA expected rates, risk-adjusted rates for respiratory failure, previously higher in the VA, became significantly higher in the private sector per 1000 hospitalizations at risk; 95% CIs did not overlap. The rates for pneumothorax and fracture were no longer significantly different across the 2 settings (see Table 3). Limiting the number of coding slots in VA administrative data to a total of 10 diagnoses and 6 procedures significantly reduced rates for all PSIs except pneumothorax, yielding comparable rates for fracture, derangement, respiratory failure, and dehiscence between the VA and private sector. However, for the other sensitivity analyses (ie, matching DRGs, controlling for the principal diagnosis, applying the cleaning period, and excluding dual users with different utilization patterns across settings), the research team did not find significantly different results compared to the baseline analyses. (Data are not shown.)

Discussion

This study compared the risk-adjusted rates for 11 PSIs between the VA and the private sector among veteran dual users of inpatient care. By limiting comparisons to this population, differences in patient case mix were reduced across settings. It was found that the PSI rates and the likelihood of developing individual PSI events varied across settings, while overall likelihood of developing any safety event was lower in the VA. The sensitivity analyses illustrated the influence of coding variations between settings on differences in PSI rates.

As expected, dual users were generally older and sicker than VA-only users because veterans must be either age 65 or older or disabled in order to qualify for Medicare coverage. Once the sample was limited to dual users, patient characteristics became relatively comparable between VA and Medicare hospitalizations. This finding is consistent with the previous literature.¹¹ However, the prevalence of specific comorbidities, such as congestive heart failure and chronic lung disease, was significantly higher among dual users hospitalized in the private sector than in the VA. Theoretically, because chronic comorbidities generally do not resolve and remain “chronic,” they should have been coded consistently across settings during the study period. Thus, these differences may indicate either issues related to potential upcoding in the private sector because of financial incentives, as discussed in the literature,¹⁹ or they may be related to the specific medical reasons at the time of hospitalization. For example, a patient could be admitted to a VA hospital for an appendectomy, and although he also had chronic lung disease, it happened to be well controlled at the time of admission and remained so during the hospitalization. It is possible that a VA coder would not code the chronic lung disease because it did not manifest during the hospitalization, although future efforts to study coding practices are needed to confirm this.

Variations were found in PSI risk-adjusted rates, as well as in the likelihood of experiencing a PSI event, between the VA and private sector. Specifically, all baseline and sensitivity analyses found lower rates for ulcer, CR-BSIs, and sepsis in the VA than in the private sector. These results are consistent with previous studies^10,11 and support the VA’s long-standing quality improvement efforts in these areas. Since 1991, the VA has used risk-adjusted rates of pressure ulcer development as a system-wide outcome measure to monitor hospital performance²⁵; it has continued to support research activities that explore best practices aimed at prevention of pressure ulcers in the inpatient care setting.^26-28 Similarly, in 1994 VA established the National Surgical Quality Improvement Program (NSQIP, now called VASQIP) to reduce surgical mortality and postoperative complications (eg, procedure-related infections); national 30-day postoperative mortality and morbidity rates decreased by 27% and 45%, respectively, from 1993 to 2000.^29
-31 Given this success in the VA, a similar voluntary program was implemented in the private sector by the American College of Surgeons (ACS-NSQIP) in 2004 and is being increasingly adopted.³² However, although suggestive, these findings do not necessarily prove causality between quality improvement initiatives and lower PSI rates.

Among the remaining 8 PSIs, baseline analyses showed that the risk-adjusted rates for 7 were significantly higher in the VA than the private sector; however, these higher rates were confirmed for only 2 PSIs (hemorrhage/hematoma and puncture/laceration) after sensitivity analyses were conducted. This suggests that the baseline findings must be interpreted cautiously. First, as discussed above, dual users were more likely to receive diagnosis codes for certain comorbidities when hospitalized in the private sector than the VA. These coding differences may explain, in part, the higher expected PSI rates and thus lower risk-adjusted rates in the private sector found in this study. After controlling for this “artificial” difference in case mix, the risk-adjusted rates for 3 PSIs (ie, respiratory failure, pneumothorax, fracture) were no longer higher in the VA.

Second, coding is very likely to be incomplete because of the limited slots for coding secondary diagnoses and procedures in the Medicare claims data²⁰; this may explain some of the differences in patient outcomes.²¹ Coders in the private sector may be forced to choose codes reflecting major diagnoses and treatments, particularly if the patient is severely ill, has multiple comorbidities, and receives numerous procedures during the hospitalization. Safety events may not be coded in the Medicare data, in particular those that do not cause serious harm to patients or do not need a reparative procedure (eg, minor accidental puncture). The possibility of incomplete coding will affect primarily the numerator of the PSIs, yielding fewer flagged PSI events in the private sector. As confirmed by the sensitivity analyses, the sheer number of diagnosis and procedure codes permitted in the VA was one of the major reasons why dual users had higher PSI rates in the VA than the private sector. These findings highlight the necessity of understanding system-level differences in coding practices when comparing quality of care across settings based on administrative data. Furthermore, lack of understanding about coding variations may lead to misinterpretation of PSI rates when they are used as measures of hospital performance on the CMS Hospital Compare Web site.⁹

There is another important issue to consider when comparing PSI rates between the VA and the private sector. Veterans may seek care in the VA versus the private sector for different medical reasons. For example, they may be more likely to obtain care at private sector hospitals for emergent medical conditions (eg, acute myocardial infarction, stroke).^14,22
-24 Therefore, reasons for admission and patient severity may vary systematically between the 2 settings. However, specifying the PSI denominator (ie, whether a particular hospitalization is at risk for a specific PSI) should help ensure the comparability of hospitalizations across settings because each hospitalization has to meet the same criteria to be eligible for a PSI. Furthermore, 2 of the sensitivity analyses attempted to control for the differences in conditions at admission (ie, matching DRGs between VA and Medicare hospitalizations, adjusting for principal diagnosis in regressions); these yielded similar results to the main analyses.

This study has some limitations. Administrative data are not comprehensive enough to explore all the factors that may influence dual users’ decisions about where to obtain hospital care (eg, patient satisfaction with care in one setting versus the other may play an important role that is not captured in administrative data). Future studies should use other data sources, such as patient interviews or surveys, to better understand these factors and incorporate the most relevant ones into comparisons of quality across settings. Additionally, the research team was unable to validate coding accuracy in this study because this would involve highly resource intensive chart review. Furthermore, clinical data were not available for risk adjustment, which may lead to insufficient control for disease severity. Last, the team was unable to include private sector hospitalizations of dual users reimbursed through other types of insurance coverage (eg, Medicare Advantage plans, Medicaid, private insurance) because data were unavailable.

This study has several strengths. The findings were based on a nationwide sample of veteran dual users, which mitigated differences in patient case mix between the VA and the private sector. Additionally, a set of sensitivity analyses were conducted to better understand the variation in the PSI rates and the likelihood of experiencing a PSI event across settings. The results highlight the strengths and weaknesses of using administrative data-based quality measures, such as the PSIs, to assess and compare hospital performance.

Conclusion

Despite the lower overall likelihood of developing any PSI event in the VA, risk-adjusted rates of individual PSIs varied across settings. Dual users had lower rates of ulcer and procedure-related infections (CR-BSIs and sepsis) when hospitalized in the VA, areas that long have been the VA’s focus for quality improvement efforts. After controlling for upcoding of comorbidities and limited coding slots in Medicare claims data, the rates of 2 PSIs (hemorrhage/hematoma and puncture/laceration) were found to be higher in the VA. Future studies should continue to explore quality of care differences across these 2 settings using appropriate comparison groups and better adjustment for coding variations.

Footnotes

Acknowledgements

We appreciate the assistance of Enzo Yaksic, BS, VA Boston Healthcare System, in helping with the literature review, references, and general preparation of the article for publication. We also acknowledge the programming and data analysis of Susan Loveland, MA, that was used in earlier work in obtaining Veterans’ Medicare claims data.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: VA Health Services Research and Development (HSR&D) Service, Grant No. SDR-07-002.

References

Institute of Medicine. To Err Is Human: Building a Safer Health System. Washington, DC: National Academies Press; 1999.

Agency for Healthcare Research and Quality. Quality indicators. http://www.qualityindicators.ahrq.gov. Accessed July 8, 2013.

Rosen

Itani

Cevasco

. Validating the patient safety indicators in the Veterans Health Administration: do they accurately identify true safety events? Med Care. 2012;50:74-85.

Zrelak

Sadeghi

Utter

. Positive predictive value of the Agency for Healthcare Research and Quality patient safety indicator for central line-related bloodstream infection (“selected infections due to medical care”). J Healthc Qual. 2011;33(2):29-36.

Utter

Cuny

Sama

. Detection of postoperative respiratory failure: how predictive is the Agency for Healthcare Research and Quality’s patient safety indicator? J Am Coll Surg. 2010;211:347-354.e1-29.

White

Sadeghi

Tancredi

. How valid is the ICD-9-CM based AHRQ patient safety indicator for postoperative venous thromboembolism? Med Care. 2009;47:1237-1243.

Utter

Zrelak

Baron

. Positive predictive value of the AHRQ accidental puncture or laceration patient safety indicator. Ann Surg. 2009;250:1041-1045.

Henderson

Recktenwald

Reichley

. Clinical validation of the AHRQ postoperative venous thromboembolism patient safety indicator. Jt Comm J Qual Patient Saf. 2009;35:370-376.

Centers for Medicare and Medicaid Services. Hospital compare. http://www.medicare.gov/hospitalcompare/?AspxAutoDetectCookieSupport=1. Accessed July 8, 2013.

10.

Rosen

Rivard

Zhao

. Evaluating the patient safety indicators: how well do they perform on Veterans Health Administration data? Med Care. 2005;43:873-884.

11.

Weeks

West

Rosen

Bagian

. Comparing measures of patient safety for inpatient care provided to veterans within and outside the VA system in New York. Qual Saf Health Care. 2008;17:58-64.

12.

Rivard

Elixhauser

Christiansen

Shibei

Rosen

. Testing the association between patient safety indicators and hospital structural characteristics in VA and nonfederal hospitals. Med Care Res Rev. 2010;67:321-341.

13.

VA Information Resource Center. Medicare Provider Analysis and Review (MEDPAR) Record—Data dictionary. http://www.ccwdata.org/cs/groups/public/documents/document/wls_ucm1-000884.pdf. Accessed July 8, 2013.

14.

Petersen

Byrne

Daw

Hasche

Reis

Pietz

. Relationship between clinical conditions and use of Veterans Affairs health care among Medicare-enrolled veterans. Health Serv Res. 2010;45:762-791.

15.

Agency for Healthcare Research and Quality. Guide to Patient Safety Indicators (Version 3.1). Rockville, MD: Agency for Healthcare Research and Quality; 2007.

16.

Agency for Healthcare Research and Quality. AHRQ Quality Indicator: National Quality Forum (NQF) endorsed composite and individual measures. http://www.qualityindicators.ahrq.gov/Downloads/Modules/V44/Module%20NQF%20Endorsement%20V4.4.pdf. Accessed April 5, 2013.

17.

Rivard

Elwy

Loveland

. Applying patient safety indicators (PSIs) across health care systems: achieving data comparability. In: Henriksen

Battles

Marks

, eds. Advances in Patient Safety: From Research to Implementation (Volume 2: Concepts and Methodology). Rockville, MD: Agency for Healthcare Research and Quality; 2005.

18.

Agency for Healthcare Research and Quality. HCUP Comorbidity Software, Version 3.5. Rockville, MD: Agency for Healthcare Research and Quality; October 2011.

19.

Silverman

Skinner

. Medicare upcoding and hospital ownership. J Health Econ. 2004;23:369-389.

20.

Zhan

Miller

MR.

Administrative data based patient safety research: a critical review. Qual Saf Health Care. 2003;12(suppl 2):ii58-ii63.

21.

Iezzoni

. Data Sources and Implications: Administrative Database. Risk Adjustment for Measuring Health Outcomes. 2nd ed. Chicago, IL: Health Administration Press; 1997:169-242.

22.

Shen

Findley

Maney

. Department of Veterans Affairs–Medicare dual beneficiaries with stroke: where do they get care? J Rehabil Res Dev. 2008;45:43-51.

23.

Wright

Daley

Fisher

Thibault

. Where do elderly veterans obtain care for acute myocardial infarction: Department of Veterans Affairs or Medicare? Health Serv Res. 1997;31:739-754.

24.

Carey

Montez-Rath

Rosen

Christiansen

Loveland

Ettner

. Use of VA and Medicare services by dually eligible veterans with psychiatric problems. Health Serv Res. 2008;43:1164-1183.

25.

Berlowitz

Halpern

. Evaluating and improving pressure ulcer care: the VA experience with administrative data. Jt Comm J Qual Improv. 1997;23:424-433.

26.

Berlowitz

Frantz

. Implementing best practices in pressure ulcer care: the role of continuous quality improvement. J Am Med Dir Assoc. 2007;8(3 suppl):S37-S41.

27.

Berlowitz

. Striving for six sigma in pressure ulcer care. J Am Geriatr Soc. 2003;51:1320-1321.

28.

Saliba

Rubenstein

Simon

. Adherence to pressure ulcer prevention guidelines: implications for nursing home quality. J Am Geriatr Soc. 2003;51:56-62.

29.

Khuri

Henderson

Daley

. Successful implementation of the Department of Veterans Affairs’ national surgical quality improvement program in the private sector: the patient safety in surgery study. Ann Surg. 2008;248:329-336.

30.

Rowell

Turrentine

Hutter

Khuri

Henderson

. Use of national surgical quality improvement program data as a catalyst for quality improvement. J Am Coll Surg. 2007;204:1293-1300.

31.

Khuri

. Safety, quality, and the national surgical quality improvement program. Am Surg. 2006;72:994-998.

32.

American College of Surgeons. NSQIP history. http://site.acsnsqip.org/program-specifics/nsqip-history/. Accessed July 8, 2013.