Inpatient Care Experiences Differ for VA and Non-VA Hospitals,With Different Patterns by Health,Race,and Ethnicity

Abstract

This article estimates differences and difference-in-differences in patient experiences for Veterans Health Administration (VA) compared to non-VA patients in 2017, when there was concern about the health quality of VA hospitals, and in 2021, the second year of the COVID-19 pandemic, both overall, and for specific patient groups. We used data from the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey. In 2017, HCAHPS performance was somewhat better for non-VA than for VA hospitals, with Care Transition being the only measure for which VA hospitals performed better on average. By 2021, HCAHPS performance was better for VA than for non-VA hospitals for all but two measures (Quietness and Discharge Information), for which there were no differences from non-VA hospitals. In 2017, the VA provided worse experiences than non-VA hospitals for Black and poor-health patients; in 2021, VA hospitals outperformed non-VA hospitals for these, and all patient subgroups examined.

Keywords

hospital quality patient experiences disparities health differences

Introduction

Since 2006, the Centers for Medicare & Medicaid Services (CMS) has administered the Hospital Consumer Assessment of Health Care Providers and Systems (HCAHPS) Survey. HCAHPS provides a standardized survey instrument and data collection methodology for measuring inpatient experiences and enables valid comparisons to be made across hospitals. To enable “apples-to-apples” comparisons to support consumer choice, it was necessary to introduce this standard measurement approach (hcahpsonline.org).

The Veterans Health Administration (VA) is one of the United States’ largest managed care organizations and has been a leader in developing innovative care across settings, including hospitals (Farrell et al., 2023). Congress, veteran’s groups, and the media have expressed concerns about access and quality of care in VA settings (Anhang Price et al., 2018). The VA originally collected data on VA outpatient care experiences through the VA Survey of Health Care Experiences of Patients, which included domains from the Consumer Healthcare Providers and Systems (CAHPS) program. Starting in 2017, the VA began administering HCAHPS, generally following the official HCAHPS protocols.

New Contribution

Our paper makes two new contributions. Despite serving a patient population with substantial disease burden (Selim et al., 2004), prior research shows that VA hospital quality of care generally compares favorably to non-VA quality of care (Anhang Price et al., 2018; Apaydin et al., 2023; Blegen et al., 2023; Trivedi et al., 2011; Yoon et al., 2023) although often without specific examination of groups highly represented in VA care. Our first contribution is to examine differences in VA and non-VA hospital experiences for two such groups, the less healthy (vis-à-vis the healthier) and Black patients (vis-à-vis White patients). Our second contribution is that we describe changes in difference in patient experiences for VA compared with non-VA patient groups over time. Specifically, we examine differences in 2017 (when the VA Patient Experiences Office initiated activities to improve VA patient experiences nationwide and the VA started following official HCAHPS protocols) and in 2021, the second year of the COVID-19 pandemic. In the first two years of the COVID-19 pandemic, patient experiences in non-VA hospitals declined (Elliott et al., 2023) following an 11-year period of steady, if at times, slow, improvement (Beckett, Quigley, Cohea, et al., 2024) but change in VA hospital patients’ experience during the pandemic has not been examined.

Method

Ethics Approval: This cohort study was approved by the institutional review board at RAND; the requirement for written informed consent was waived because only deidentified data were obtained.

The HCAHPS survey consists of items used to construct 10 patient experience measures and an “About You” section asking for demographic information, some of which is used to adjust for patient-mix differences across hospitals. HCAHPS is administered to random samples of adult inpatients with an overnight stay in the medical, surgical, and maternity service lines (Centers for Medicare & Medicaid Services, 2024a, 2024b; Giordano et al., 2010). The survey is initiated between 48 hours and 42 days after discharge using one of four approved modes of administration: Mail Only, Telephone Only, Mixed Mode (mail with telephone follow-up of non-respondents), and active Interactive Voice Response (Elliott et al., 2009). The VA only uses Mail Only administration. Because the VA sample included no maternity patients (vs. 10%-12% of respondents for non-VA hospitals) and <0.1% of patients less than 25 years of age (vs. 2%-3% for non-VA respondents), we restricted the analytic sample to surgical and medical patients aged 25 or older.

We constructed an HCAHPS “top-box” summary score (HCAHPS-SS) that combined 10 HCAHPS measures: six multi-item composite measures each weighted 1.0 (Communication with Nurses, Communication with Doctors, Staff Responsiveness, Communication about Medicine, Care Transition, and Discharge Information) and 4 single-item measures that were each weighted 0.5 (Cleanliness, Quietness, Overall Hospital Rating, Hospital Recommendation) for 2017 and 2021. Survey response options were “Never,” “Sometimes,” “Usually,” or “Always” for all measures except Discharge Information (options were “Yes” or “No”), Care Transition (“Strongly disagree,” “Disagree,” “Agree,” or “Strongly agree”), Hospital Recommendation (“Definitely no,” “Probably no,” “Probably yes,” or “Definitely yes”), and Hospital Rating (response options ranged from 0 [“Worst possible hospital”] to 10 [“Best possible hospital”]). Top-box scoring, which is the proportion of most positive responses (except for Hospital Rating, for which 9 or 10 correspond to a top-box response), was used for all HCAHPS measures and the HCAHPS-SS. All reported scores were adjusted for survey mode and patient mix (including age and service line) for comparability.

We compared VA and non-VA patient characteristics in each year. In our analysis, VA-contracted hospitals are classified as non-VA hospitals since they serve a combination of veteran and non-veteran patients. To make results comparable across hospitals and over time, HCAHPS data were adjusted both for the mode of survey administration and for patient characteristics known to affect survey responses but not under the control of the hospital (Cefalu et al., 2021; Elliott et al., 2009). By year, we compared VA/non-VA differences for HCAHPS-SS and for specific individual HCAHPS measures within four patient groups defined by self-reported overall health and race-and-ethnicity (poor-health, excellent-health, Black, and White) using mixed-effect linear regression models with an indicator of being a VA hospital and hospital random effects.

Results

In both 2017 and 2021, there were large differences in the characteristics of patients in VA and non-VA hospitals (Table 1). In 2021, VA patients were more often male (95% vs. 44% for non-VA), in fair or poor mental health (22% vs. 12% non-VA), in fair or poor physical health (40% vs. 27% non-VA), and age 65 years or older (81% vs. 60% non-VA).

Table 1.

Characteristics of VA Hospital Patients and Non-VA Hospital Patients, 2017 and 2021.

	2017		2021
Patient characteristic	VA	Non-VA	VA	Non-VA
Number of patients	41,063	2,597,010	48,646	2,047,386
Number of hospitals	127	4,500	122	4,429
Characteristic
Education
No high school	3.6%	4.8%	2.2%	4.1%
Some high school	7.6%	8.2%	6.0%	7.1%
High school graduate	35.7%	32.2%	34.8%	31.7%
Some college	37.2%	29.5%	37.9%	30.1%
Four-year college degree	8.1%	11.7%	9.8%	12.7%
More than 4-year college degree	7.9%	13.6%	9.3%	14.3%
Overall health
Excellent	6.8%	10.3%	5.9%	9.0%
Very good	16.3%	26.0%	17.6%	24.3%
Good	33.9%	35.5	36.4%	37.4%
Fair	31.6%	21.1%	30.9%	23.0%
Poor	11.4%	7.1%	9.1%	6.3%
Mental health
Excellent	20.8%	28.2%	19.0%	25.3%
Very good	27.3%	33.7%	28.6%	33.8%
Good	29.4%	26.9%	30.8%	28.6%
Fair	18.3%	9.5	18.0%	10.6%
Poor	4.2%	1.8%	3.6%	1.8%
Age
25–34	0.4%	2.9%	0.3%	2.4%
35–44	0.9%	4.4%	1.0%	4.1%
45–54	4.1%	9.9%	3.1%	8.0%
55–64	19.0%	21.4%	14.3%	18.9%
65–74	46.1%	30.2%	43.6%	30.7%
75–85	19.4%	22.1%	27.3%	25.4%
85+	10.1%	9.1%	10.5%	10.6%
Service line and sex
Female medical	3.1%	31.2%	4.0%	32.6%
Male medical	72.3%	26.5%	76.3%	30.5%
Female surgical	1.4%	21.8%	1.2%	18.2%
Male surgical	23.2%	20.6%	18.5%	18.7%
Preferred language
Chinese	0.0%	0.3%	0.0%	0.3%
English	98.4%	94.5%	98.0%	93.7%
Spanish	1.4%	3.9%	1.7%	4.5%
Other	0.2%	1.3%	0.2%	1.4%
Race and ethnicity
Asian American, Native Hawaiian or Other Pacific Islander	0.5%	2.0%	0.8%	2.5%
American Indian or Alaska Native	1.0%	0.7%	0.8%	0.7%
Black	13.8%	7.8%	16.2%	7.8%
Hispanic	6.5%	8.7%	7.2%	9.9%
Multiracial	2.3%	3.1%	1.9%	1.9%
White	76.0%	77.6%	73.1%	77.3%

Note. 2017 (2021) samples exclude 357,530 (357,460) non-VA maternity and/or patients <25 years and 14 (16) VA patients <25 years.

As shown in Table 2, in 2017, HCAHPS performance, averaged across all dimensions using HCAHP-SS, was somewhat better for non-VA than for VA hospitals, though the difference varied by measure, with Care Transition as the only measure for which VA hospitals performed better on average (+3.2 percentage points, pp); the VA lagged the most for Quietness (−7.2pp) and Cleanliness (−5.2pp). By 2021, HCAHPS performance overall and for 10 specific measures was better for VA than non-VA hospitals. There were no differences between VA and non-VA hospitals for the two remaining measures (Quietness and Discharge Information). As indicated by positive and statistically significant differences-in-differences, VA hospitals’ performance improved between 2017 and 2021 relative to non-VA hospitals for HCAHPS-SS and all 12 specific measures. This difference-in-differences corresponds to an increase in VA performance while non-VA performance fell during the COVID-19 pandemic. In 2017, the mean HCAHPS-SS was 70.0 for VA hospitals but 71.5 for non-VA hospitals. By 2021, mean HCAHPS-SS increased to 72.7 for VA and declined for non-VA hospitals (to 69.5; results not shown).

Table 2.

Estimate of Differences of VA Hospitals From Non-VA Hospitals, 2017 and 2021.

	2017		2021		Difference in differences
HCAHPS measure	Est/%	SE	Est/%	SE	Est/%	SE
HCAHPS-SS	−0.014**	0.005	0.030***	0.006	0.044***	0.008
Communication with nurses	−0.039***	0.005	0.010*	0.005	0.050***	0.007
Communication with doctors	−0.027***	0.000	0.017**	0.005	0.044***	0.005
Staff responsiveness	−0.029**	0.008	0.041***	0.009	0.071***	0.013
Communication about medicines	0.001	0.005	0.060***	0.006	0.058***	0.008
Cleanliness	−0.052***	0.007	0.015*	0.007	0.068***	0.010
Quietness	−0.072***	0.012	−0.019	0.010	0.053***	0.016
Discharge information	−0.017***	0.003	0.000	0.004	0.017***	0.005
Care transition measure	0.032***	0.006	0.062***	0.006	0.030***	0.009
Overall rating of hospital	−0.036***	0.008	0.028**	0.008	0.064***	0.011
Hospital recommendation	−0.034***	0.008	0.025**	0.009	0.059***	0.013

***

p < .001. **p < .01. *p < .05.

Figure 1 shows adjusted VA vs. non-VA differences in HCAHPS-SS overall and for poor-health patients, excellent-health patients, Black patients, and White patients. In 2017, the VA provided worse experiences than non-VA hospitals for Black and poor-health patients; in 2021, VA hospitals outperformed non-VA hospitals for all four patient groups examined.

Figure 1.

Difference of VA Hospitals From Non-VA Hospitals HCAHPS Summary Score, All Patients, Excellent-Health, Poor-Health, Black, and White Patients, 2017 and 2021.

Discussion

In contrast to 2017, in 2021 VA hospitals provided better patient care experiences than non-VA hospitals overall, for all HCAHPS measures, and for all patient groups assessed. The degree of improvement in VA hospitals over 4 years (2.7 pp) is remarkable under any circumstances (Elliott et al., 2015), but especially since this period included the first 2 years of the COVID-19 pandemic, during which the performance of non-VA hospitals declined (Elliott et al., 2023).

Our finding that patients reported overall worse experiences in VA versus non-VA hospitals in 2017 is broadly similar to prior findings. One study found that in 2014, VA hospitals scored higher for four of eight individual HCAHPS measures than non-VA hospitals and lower for the other four measures (Anhang Price et al., 2018). However, a second, small study found comparable surgical patient experiences in 2018 in VA and other hospitals in three cities (Eid et al., 2020). Ours is the only study of which we are aware that compares VA and non-VA patient experiences during the stress associated with the COVID-19 pandemic. Our analysis is limited in that it only considers average VA and non-VA performance; there is substantial heterogeneity of hospitals within these two categories in terms of payer mix, location, and bed size, each of which is associated with HCAHPS performance (Beckett, Quigley, Lehrman, et al., 2024).

Our results differ somewhat from those comparing veterans’ outpatient care experiences at VA facilities and in community settings. In the outpatient setting in 2016, veterans’ experiences in VA facilities were better than veterans’ experiences in community settings for communication, coordination, and provider rating but not for access in 2016; in addition, patient experience scores for all VA patients improved slightly from 2016 through 2017, but the gap between VA facilities and community settings did not change (Vanneman et al., 2020). In contrast, our baseline inpatient findings for 2017 showed higher ratings for non-VA care. In addition, our study can explore data from 2021 to show that VA care improved while non-VA care declined during the COVID-19 pandemic, with the VA surpassing other hospitals overall. Differences in setting, patient populations, and timing between our study and the earlier outpatient study make it difficult to know why the results differed.

Several changes occurred within the VA health care system following the 2014 waitlist crisis (Rubin, 2014), which may have contributed to the positive trends in patient care in VA hospitals vis-à-vis non-VA hospitals. First, VA patient care culture shifted, resulting in the establishment of a Veterans Experience Office that reports directly to the Secretary of Veterans Affairs, the designation of patient experiences officers in every facility, the development of performance plans inclusive of patient experience goals, and the identification of best practices and key drivers of good patient experience. Best practices were developed into toolkits to facilitate easy adoption at all VA sites. To the extent that quality improvement efforts resulted in more uniform care, Black patients and those in poor health, patient groups who are often underserved, may have benefited especially.

Second, the VA Maintaining Internal Systems and Strengthening Integrated Outside Networks (MISSION) Act was implemented in 2019 to expand access and provider choice for VA patients by broadening eligibility for VA-paid community care from that in the 2014 Choice and Accountability Act. This shifted some VA patients to non-VA providers and was associated with reductions in VA hospitalization volume and an increase in VA-funded community hospitalizations (Axon et al., 2024).

Our analysis was limited in that we had no information about patient rurality, which is associated with VA outpatient care experiences (Davila et al., 2021) and with general hospital patient experiences (Beckett, Quigley, Lehrman, et al., 2024). Future research should assess whether the improved patient experiences in VA hospitals are sustained post-pandemic, what changes in care processes (such as patient-to-staff ratios), or unmeasured patient-mix characteristics (such as patient acuity) may have differed in community hospitals and VA hospitals, and whether the quality improvement strategies differed in nature and effectiveness. If effective and transferable strategies are identified, they could be disseminated to improve patient experience for non-VA hospitals.

Footnotes

Acknowledgements

The authors thank Biayna Darabidian, MPP, Katherine Osby, BA, and Lauren Lakritz, MAIA, for their help with preparing the manuscript.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: James Schaefer is an employee the VA, whose performance is being evaluated; Debra Saliba is also affiliated with the VA; neither had a role in the design of the analysis. Beyond this, the authors declare no conflict of interest. However, William G. Lehrman and Elizabeth Goldstein are employees of the sponsoring agency, the Centers for Medicare & Medicaid Services. The ideas expressed in this article are solely those of the authors, and no endorsement of these views or opinions by the authors’ institutions or funders is expressed or implied.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the Centers for Medicare & Medicaid Services under contract 75FCMC22F0075 to the Health Services Advisory Group.

Ethics Approval

This cohort study was approved by the institutional review board at RAND; the requirement for written informed consent was waived because only deidentified data were obtained.

Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

ORCID iD

Marc N. Elliott

Data Availability

The data in this study are available via a DUA with the Centers for Medicare & Medicaid Services.

References

Anhang Price

Sloss

E. M.

Cefalu

Farmer

C. M.

Hussey

P. S

. (2018). Comparing quality of care in Veterans affairs and non-veterans affairs settings. Journal of General Internal Medicine, 33(10), 1631–1638. https://doi.org/10.1007/s11606-018-4433-7

Apaydin

E. A.

Paige

N. M.

Begashaw

M. M.

Larkin

Miake-Lye

I. M.

Shekelle

P. G.

(2023). Veterans Health Administration (VA) vs. non-VA healthcare quality: A systematic review. Journal of General Internal Medicine, 38(9), 2179–2188.

Axon

R. N.

Ward

Mohamed

Pope

Stephens

Mauldin

P. D.

Gebregziabher

(2024). Trends in Veteran hospitalizations and associated readmissions and emergency department visits during the MISSION Act era. Health Services Research, 59(5), e14332.

Beckett

M. K.

Quigley

D. D.

Cohea

C. W.

Lehrman

W. G.

Russ

Giordano

L. A.

Goldstein

Elliott

M. N.

(2024). Trends in HCAHPS Survey Scores, 2008–2019: A quality improvement perspective. Medical Care, 62(6), 416–422.

Beckett

M. K.

Quigley

D. D.

Lehrman

W. G.

Giordano

L. A.

Cohea

C. W.

Goldstein

E. H.

Elliott

M. N.

(2024). Interventions and hospital characteristics associated with patient experience: An update of the evidence. Medical Care Research and Review, 81(3), 195–208.

Blegen

Salzman

Begashaw

M. M.

Ulloa

J. G.

Girgis

Shekelle

Maggard-Gibbons

(2023). Comparing quality of surgical care between the US Department of Veterans Affairs and non-veterans affairs settings: A systematic review. Journal of the American College of Surgeons, 237(2), 352–361.

Cefalu

Elliott

M. N.

Hays

R. D.

(2021). Adjustment of patient experience surveys for how people respond. Medical Care, 59(3), 202–205.

Centers for Medicare & Medicaid Services. (2024a). CAHPS Hospital Survey. http://www.hcahpsonline.org

Centers for Medicare & Medicaid Services. (2024b). HCAHPS. https://www.hcahpsonline.org/

10.

Davila

Rosen

A. K.

Beilstein-Wedel

Shwartz

Chatelain

Jr. Gurewich

(2021). Rural veterans’ experiences with outpatient care in the veterans health administration versus community care. Medical Care, 59, S286–S291.

11.

Eid

M. A.

Barnes

J. A.

Trooboff

S. W.

Goodney

P. P.

Wong

S. L.

(2020). A comparison of surgical quality and patient satisfaction indicators between VA hospitals and hospitals near VA hospitals. Journal of Surgical Research, 255, 339–345. https://doi.org/10.1016/j.jss.2020.05.071

12.

Elliott

M. N.

Beckett

M. K.

Cohea

C. W.

Lehrman

W. G.

Cleary

P. D.

Giordano

L. A.

Russ

Goldstein

E. H.

Fleisher

L. A.

(2023). Changes in patient experiences of hospital care during the COVID-19 pandemic. JAMA Health Forum, 4(8), e232766. https://doi.org/10.1001/jamahealthforum.2023.2766

13.

Elliott

M. N.

Cohea

C. W.

Lehrman

W. G.

Goldstein

E. H.

Cleary

P. D.

Giordano

L. A.

Beckett

M. K.

Zaslavsky

A. M.

(2015). Accelerating improvement and narrowing gaps: Trends in patients’ experiences with hospital care reflected in HCAHPS public reporting. Health Services Research, 50(6), 1850–1867. https://doi.org/10.1111/1475-6773.12305

14.

Elliott

M. N.

Zaslavsky

A. M.

Goldstein

Lehrman

Hambarsoomians

Beckett

M. K.

Giordano

(2009). Effects of survey mode, patient mix, and nonresponse on CAHPS^® hospital survey scores. Health Services Research, 44(2p1), 501–518.

15.

Farrell

T. W.

Volden

T. A.

Butler

J. M.

Eleazer

G. P.

Rupper

R. W.

Echt

K. V.

Shaughnessy

Supiano

M. A.

(2023). Age-friendly care in the Veterans Health Administration: Past, present, and future. Journal of the American Geriatrics Society, 71(1), 18–25.

16.

Giordano

L. A.

Elliott

M. N.

Goldstein

Lehrman

W. G.

Spencer

P. A.

(2010). Development, implementation, and public reporting of the HCAHPS survey. Medical Care Research and Review, 67(1), 27–37.

17.

Rubin

(2014). Testimony on VA Waiting List Scandal. JAMA, 312(14), 1389–1389. https://doi.org/10.1001/jama.2014.13714

18.

Selim

A. J.

Berlowitz

D. R.

Fincke

Cong

Rogers

Haffer

S. C.

Ren

X. S.

Lee

Qian

S. X.

Miller

D. R.

Spiro

3rd Selim

B. J.

Kazis

L. E.

(2004). The health status of elderly veteran enrollees in the Veterans Health Administration. Journal of the American Geriatrics Society, 52(8), 1271–1276.

19.

Trivedi

A. N.

Matula

Miake-Lye

Glassman

P. A.

Shekelle

Asch

(2011). Systematic review: Comparison of the quality of medical care in Veterans Affairs and non-Veterans Affairs settings. Medical Care, 49(1), 76–88. https://doi.org/10.1097/MLR.0b013e3181f53575

20.

Vanneman

M. E.

Wagner

T. H.

Shwartz

Meterko

Francis

Greenstone

C. L.

Rosen

A. K.

(2020). Veterans’ experiences with outpatient care: Comparing the Veterans Affairs system with community-based care: Study compares veterans’ experiences in VA-delivered and community-based outpatient care. Health Affairs, 39(8), 1368–1376.

21.

Yoon

Phibbs

C. S.

Ong

M. K.

Vanneman

M. E.

Chow

Redd

Kizer

K. W.

Dizon

M. P.

Wong

Zhang

(2023). Outcomes of veterans treated in veterans affairs hospitals vs non-veterans affairs hospitals. JAMA Network Open, 6(12), e2345898.