Setting performance benchmarks for stroke care delivery: Which quality indicators should be prioritized in quality improvement;an analysis in 500,331 stroke admissions

Abstract

Background and aim

Benchmarking is a management approach for implementing best medical practices at the lowest cost. The objectives of this study were to set achievable performance benchmarks for individual quality indicators to determine the predicted quality achievement related to better adherence, and to select optimal quality indicators for improving the quality of acute ischemic stroke care.

Methods

We analyzed data on 500,331 patients diagnosed with acute ischemic stroke who were treated at 518 hospitals in China from January 2011 to May 2017. The primary outcome was independence (modified Rankin Scale score ≤2) at discharge. Data-driven achievable benchmarking used the “pared-mean” approach to set objective performance targets. Hierarchical logistic regression models were employed to evaluate the process–outcome association, as well as the predicted quality improvement if all hospitals were to operate at the benchmark level.

Results

Of the overall population, 64.01% were independent patients at discharge. The performance benchmarks were >90% for most of the quality indicators. After adjusting for patient-level and hospital-level characteristics and unifying hospital performance to the benchmark level, the quality indicators with high increase in both overall adherence rate and independence rate were thrombolytic therapy, anticoagulant therapy, venous thrombosis prophylaxis.

Conclusions

Performance targets for three acute treatments, including thrombolytic therapy, anticoagulant therapy, venous thrombosis prophylaxis, could best motivate improvements in both overall adherence rate and independence rate at discharge. The finding suggests that the above three types of acute treatment should be given priority to improve the quality of acute ischemic stroke care.

Keywords

Performance benchmark stroke care delivery quality indicators quality improvement

Background

Stroke is a crippling and potentially deadly condition affecting millions of people worldwide each year.^1,2 Preventing stroke and improving the quality of stroke care have been global health priorities.^3–5 Although new therapies are needed to meet the challenge of improving the quality of stroke care, better application of currently available therapies may offer more immediate results in terms of saving lives and improving function. The implementation of and adherence to guideline-recommended treatments form a foundation for quality improvement and may serve as a marker of quality of care.⁶

Benchmarking is a management approach for implementing best medical practices at the lowest cost.⁷ It provides performance targets for improvement and promotes emulation of those providers achieving “best practice.”⁸ However, it is a huge challenge to ensure that guideline-recommended treatments be more widely adopted and that frontline professionals be more involved by using benchmarking processes. How do we set objective, reproducible, and attainable performance benchmarks on the way to improving the quality of stroke care? Since a complex mix of factors can influence the implementation of and the adherence to quality indicators (QIs), setting “perfect” targets for process performance is unrealistic and inappropriate. Therefore, an important goal of the benchmark setting is to aid in the spread of “excellence practices” by a few superior providers until they become “average performance” by the majority.

Although adherence to guideline-recommended treatments has been demonstrated to improve patient outcomes, the strength, magnitude, and statistical significance of the positive effects on clinical outcomes vary across individual QIs.^9–11 Before the benchmark intervention, we need to speculate the net gains in quality improvement of performance benchmarks setting and identify QIs that should be prioritized in improving the quality of stroke care.

The purpose of this study was to set achievable benchmarks for a broad spectrum of QIs based on long-term medical records on stroke care practices across China and to determine the predicted quality achievement in terms of process-adjusted positive outcomes if all hospitals were to operate at the benchmark level and to select optimal QIs for improving the quality of stroke care. To predict the composite quality achievement with benchmarking, we calculated the hospital process composite performance (HPCP).

Methods

Data source

The analyses were based on data from the Medical Care Quality Management and Control System for Specific Diseases of China, an ongoing voluntary and continuous web-based registry. The diseases in the registry included stroke, heart failure, acute myocardial infarction, cesarean section, community-acquired pneumonia (adults or children), hip/knee replacement, coronary artery bypass grafting, chronic obstructive pulmonary disease, perioperative infection prevention, and perioperative prevention of deep venous thrombosis (DVT). The system was designed to collect and manage data on QIs and provide hospitals with real-time quality-of-care reports and data validation. Since 2015, a national report on the services, quality, and safety in medical care system has been published annually based on data from the system. Each QI has an explicit definition in the reports. The system collected data from secondary or tertiary hospitals in the 31 provinces, autonomous regions, and municipalities in China. It represents more than half of all hospitalizations in tertiary hospitals nationwide. For stroke, hospitals were restricted to those with emergency departments and neurological wards that admitted patients with stroke and had the capacity to administer recombinant tissue plasminogen activator (rt-PA).

Because the data were fully anonymized and used primarily for hospital care quality management, ethical approval was not required under China research governance arrangements. Individual patient consent was waived as it was determined that this study does not meet the regulatory definition of human subject research.

Case reporting process

Data were extracted from the inpatient medical records, drug charts, discharge summary, and assessment sheets. Collectors could request additional materials such as laboratory reports stored on the computer, if they were missing from the clinical record. If the information still could not be found, it was taken as real missing data. The information collected by the system included baseline demographic characteristics, diagnostic testing, detailed medication history, health care utilization, in-hospital outcomes, and satisfaction with hospital service quality.

Patients with acute ischemic stroke (AIS) were diagnosed by the attending physician. Collectors identified the target patients from medical records according to the International Classification of Diseases version 10, diagnosis codes. Patients with other cerebrovascular diseases, such as hemorrhagic stroke, transient ischemic attack, cerebral venous sinus thrombosis, or non-cerebrovascular diseases, were not reported. Eligible record IDs were loaded into a Microsoft Excel spreadsheet and were arranged randomly and then selected consecutively. The proportion of cases selected for data reporting should not be less than 20%. The collectors were required to complete the data reporting at the end of each month.

Data reporting was conducted by trained clinical data collectors using standardized definitions. Each participating hospital appointed a surgeon or nurse responsible for (supervising) the data reporting in a secured web form. Before the online data reporting, collectors needed to read data reporting instruction carefully. The training process consisted of a web-based teaching segment for data reporting of five cases and an assessment process with system-generated 10 cases. When the inter-rater reliability reached 95% or more, the collectors could start data reporting, otherwise they would have a second teaching and assessment session. To minimize error caused by manual reporting, the system automatically collected the information on the front page of inpatient medical records from the hospital information system via a web-based data collection tool. Values that exceeded the expected ranges prompted error notification. The system also provided predefined logic checks to identify errors or illogical data entries. Besides, annual feedback on data quality problems and quality-of-care provided to all registered hospitals.

Study population

The analytical cohort of 500,331 patients treated at 518 hospitals was drawn from the 571,522 patients with a discharge diagnosis of AIS treated at 756 hospitals from 1 January 2011 to 31 May 2017. We restricted our sample to patients (1) who admitted in transfer; (2) without aberrant values, including outliers or incorrect reporting values; (3) whose age at diagnosis was older than 18 years; (4) who were hospitalized for more than one day but no longer than 120 days; and (5) who had outcome records at discharge. We also restricted our sample to hospitals (1) with at least 20 admissions overall; (2) with eligible patients for all QIs selected for this study. Because hospitals with a small number of patients were inclined to have perfect performance, which could inflate the benchmark level, hospitals with fewer than 20 admissions overall were excluded. Figure 1 presents the number of study flow diagram of the patient population. Although 238 hospitals and 71,191 patients were excluded according to the above-mentioned criteria, there was no significant difference between the included group and the excluded group (Suppl Table I and Table II).

QIs and outcomes

We used nine The Get With The Guidelines (GWTG)–Stroke performance measures,¹² including acute and subacute treatments. The specific definitions of QIs are listed in online-only supplementary table III. As we aimed to assess the relationship between adherence to QIs and patient in-hospital outcomes, the study excluded QIs for medication prescribed at discharge.

Acute treatments included: (1) CT/MRI scan, (2) laboratory tests, (3) electrocardiogram, (4) acute thrombolytic therapy within 3 h of symptom onset; (5) anticoagulant therapy prescribed within 48 h of admission; (6) antiplatelet therapy prescribed within 48 h of admission; (7) deep vein thrombosis prophylaxis within 48 h of admission if nonambulatory.

Subacute treatments included: (8) statins therapy during hospitalization if the low-density lipoprotein level was >2.6 mol/L; (9) dysphagia screening before any oral intake during hospitalization.

Patient outcome was assessed with the modified Rankin Scale (mRS) score at discharge.¹³ Patient outcome was classified as independence (mRS score 0–2) or dependence/death (mRS score 3–6).^14,15

Statistical analysis

Patient baseline characteristics for the study population were described in counts and percentages, mean, and standard deviation. Differences between independence group and dependence/death group were expressed using standardized differences. To assess hospital performance on stroke care, our primary measurement of adherence was presented as a proportion of the sum of patients who have been provided with care (numerator) to the total number of patients eligible for the care (denominator). Patients with clinical-care contraindication or lack of information on care eligibility were excluded from both the numerator and denominator. We divided patients into four groups according the number of coexisting conditions (0, 1, 2–3, ≥4) and calculated the adherence rates of individual QIs in each group.

Data-driven achievable benchmarks of care used the “pared-mean” approach to set objective performance targets.^8,16 We ranked hospitals in descending order according to their performance of individual QIs. Starting with the top-ranked hospital, we cumulated each hospital’s eligible patients until a subset of patients represented at least 10% of the eligible population across all hospitals. The subset was the benchmark subset. We calculated a benchmark level for each QI based on the average performance of the subset. To predict the composite benchmark effect of all QIs, we calculated the HPCP using denominator-based weights, which was defined as the ratio of the total number of documented correct care to the sum of care opportunities for all QIs.^17,18 Here, we used the same method as individual QIs to set achievable performance benchmarks for HPCP. We calculated statistical p values for the associations of the hospital structural elements with HPCP using Kruskal–Wallis H tests.

The process–outcome association was investigated using hierarchical logistic regression with patient-level and hospital-level risk factors as fixed effects and a random intercept for hospitals. The full model included the following risk factors: year and age at diagnosis, stroke subtype and severity, primary payer status, pattern of admission, comorbidities, geographic region, hospital type, hospital level, university-affiliated status, nurse-to-bed ratio, health technician-to-bed ratio, and annual outpatient volume. Hospital process performance (adherence to each QI or the HPCP) was added as a continuous predictor variable, and odds ratios (ORs) were reported per 10% increment in hospital process performance. We entered variables into the models simultaneously.

To determine the predicted quality achievement in terms of independence at discharge if all eligible hospitals were to operate at the benchmark level, we first identified hospitals in the benchmark subset. From the full covariate-adjusted hierarchical model, we considered the effect of hospital process performance as random effect. For the entire study population, we used the hierarchical model to predict the proportion of independent patients at discharge given their fixed covariates and the random effect of the benchmark subset.¹⁹ The random effect of the benchmark subset was the weighted mean of the random effect of hospitals in the subset, which was weighted by the number of eligible patients.

Relying on the process–outcome association and using self-control method, we calculated the predicted improvement of adherence rate (IAR) and the predicted improvement of independence rate (IIR). The 95% confidence intervals (95% CI) were calculated using the bootstrap method with 1000 replications. IAR was the difference between the benchmark performance level of each QI and its observed overall adherence rate. IIR was the overall increase in the proportion of independent patients after unifying hospital performance to the benchmark level.

To mitigate against potential bias due to multiple data exclusions, we conducted a sensitivity analysis. In the sensitivity analysis, we redefined the analytical population by only excluding patients who lacked vital information on care eligibility and conducted the same analyses described above. We used SAS statistical software, Version 9.3 (SAS Institute Inc., Cary, NC, USA) for all statistical analyses.

Results

Patient characteristics

Table 1 summarized the demographic characteristics of patients in the independent and dependent/dead group. Of the 500,331 patients diagnosed with AIS, 320,281 (64.01%) were independent patients at discharge. The mean age at diagnosis was 66.28 years (standard deviation, 12.38 years). More than 80% of study population were large/small artery AIS. The proportion of coexisting diseases was much higher in the independent group than in the dependent/dead group; 91,425 (18.27%) patients were complicated with other diseases, in which 68.00% (62,168 in 91,425) had only one coexisting disease. In most of QIs, the adherence rates of patients with one coexisting condition were higher than that of patients without coexisting condition, but the adherence rate decreased as the number of coexisting conditions increased (Suppl Table IV). The same trend was shown in HPCP. Annual enrollment proportions of cases did not vary much, ranging from 11.46% to 19.04% (data recorded for half of 2017).

Table 1.

Variation in patients’ characteristics between dependent/death and independent group

Characteristics	Dependence/death (mRS 3–6)	Independence (mRS 0–2)	Standardized difference
Patients, n	180,050	320,281
Age
M (SD)	66.47 (12.50)	66.17 (12.32)	−0.0242
<55	36,327 (20.18)	64,956 (20.28)	0.0025
55–65	46,135 (25.62)	84,398 (26.35)	0.0166
65–75	50,018 (27.78)	90,643 (28.30)	0.0116
>75	47,570 (26.42)	80,284 (25.07)	−0.0309
Stroke subtype
Large artery	69,429 (38.56)	120,631 (37.66)	−0.0185
Small artery	74,710 (41.99)	150,454 (46.98)	0.1005
Cardioembolic	1142 (0.63)	10,087 (3.15)	0.1859
Other determined cause	7928 (4.40)	7819 (2.44)	−0.1080
Undetermined	26,841 (14.91)	31,290 (9.77)	−0.1568
NIHSS at admission, M (SD)	3.65 (3.90)	4.41 (4.26)	0.1861
Primary payer status
Socialized medicine	9490 (5.27)	9722 (3.04)	−0.1119
Basic insurances	133,387 (74.08)	252,353 (78.79)	0.1112
Other insurances	13,657 (7.59)	22,515 (7.03)	−0.0215
Self-paying	23,516 (13.06)	35,691 (11.14)	−0.0589
Patterns of admission
Emergency	81,688 (45.37)	149,700 (46.74)	0.0275
Outpatient	98,362 (54.63)	170,581 (53.26)	−0.0275
Coexisting diseases
Diabetes mellitus	2844 (1.58)	24,477 (7.64)	0.2920
Myocardial infarction	287 (0.16)	2680 (0.84)	0.0965
Pneumonia	802 (0.45)	4510 (1.41)	0.1001
Coronary heart disease	1152 (0.64)	10,339 (3.23)	0.1889
Renal failure	259 (0.14)	1677 (0.52)	0.0663
COPD	296 (0.16)	2571 (0.80)	0.0927
Arrhythmia	866 (0.48)	5252 (1.64)	0.1135
Hematological disease	3563 (1.98)	40,754 (12.72)	0.4206
Cerebrovascular disease	2290 (1.27)	24,770 (7.73)	0.3155
Rheumatic valvular heart diseases	223 (0.12)	1922 (0.60)	0.0802
Congestive heart failure	494 (0.27)	7670 (2.39)	0.1859
Year
2011	24,588 (13.66)	32,759 (10.23)	−0.1059
2012	32,973 (18.31)	50,611 (15.80)	−0.0668
2013	34,208 (19.00)	61,032 (19.06)	0.0015
2014	26,332 (14.62)	48,749 (15.22)	0.0168
2015	25,470 (14.15)	49,938 (15.59)	0.0405
2016	24,146 (13.41)	55,895 (17.45)	0.1120
2017	12,333 (6.85)	21,297 (6.65)	−0.0080

mRS: modified Rankin Scale; NIHSS: National Institutes of Health Stroke Scale score; SD: standard deviation; COPD: chronic obstructive pulmonary disease.

Hospital performance on QIs

Eligible patients and adherence rates were listed in Table 2. The overall adherence differed greatly in different QIs, ranging from 22.81% of acute thrombolytic therapy to 98.75% of antiplatelet therapy. We also observed significant differences in hospital adherence to the QIs, there are three QIs with the interquartile range (IQR) of hospital adherence close to 50%: laboratory tests, electrocardiograph report, dysphagia screening. The overall composite performance on all QIs was 62.99%, and the median HPCP was 68.33% (IQR, 51.96–77.86%). Figure 2 shows the distribution of hospital composite performance by structural elements. Hospitals located in the eastern region of China and private hospitals have superior performance levels (P < 0.05).

Table 2.

Adherence to individual quality indicators

Quality indicators	Eligible patients, No^a	Overall Adherence, %	Hospital variation, median (IQR) %
1, Acute stroke care
QI1	Door to CT or MRI time within 45 minutes of admission
	220,662	73.69	79.17 (65.03, 89.62)
QI2	Laboratory tests within 45 min of admission
	225,308	51.83	53.54 (28.47, 78.26)
QI3	ECG report within 45 min of admission
	220,880	51.66	50.00 (25.45, 75.35)
QI4	Acute thrombolytic therapy within 3 h of symptom onset
	128,265	22.81	22.22 (16.28, 30.51)
QI5	Anticoagulant therapy within 48 h of admission
	15,216	59.07	60.00 (44.23, 75.00)
QI6	Antiplatelet therapy within 48 h of admission
	294,990	98.75	99.11 (98.00, 99.83)
QI9	Venous thrombosis prophylaxis within 48 h of admission if nonambulatory^b
	59,075	84.35	87.71 (77.65, 95.94)
2, Subacute stroke care
Q8	Statins therapy^c
	186,449	92.07	95.27 (90.12, 97.63)
QI11	Dysphagia screening before any oral intake
	398,390	47.04	60.10 (31.74, 83.97)
3, HPCP	292,9885^d	62.99	68.33 (51.96, 77.86)

QI: quality indicator; IQR: interquartile range; CT: computed tomography; MRI: magnetic resonance imaging; ECG: electrocardiograph; HPCP: hospital process composite performance.

All QIs were assessed among patients with definite indications but no documented contraindications. Observations that lacked of information about the care eligible definition were not included in the eligible population.

Eligible for patients who were not ambulating by hospital day 2.

Eligible for patients with low-density lipoprotein >2.6 mol/L, and accompanied by hypertension and diabetes.

The sum of eligible patients of all indicators.

Achievable benchmark setting and predicted quality achievement

Figure 3 described the risk-adjusted process–outcome association. After adjusting for patient-level and hospital-level characteristics, adherence to QIs showed a positive association with independence rate. Every 10% increase in HPCP score was associated with a 49.9% higher proportion of independent patients (OR = 1.499, 95% CI: 1.440–1.560).

For each QI and HPCP, we identified the benchmark subset (Table 3). Patients in the subset accounted for about 10% of the eligible populations of QIs. Except for 49.97% of acute thrombolytic therapy, the performance benchmarks of other QIs were more than 90%. The composite benchmark level of HPCP was 82.47%. After adjusting for patient-level and hospital-level characteristics and unifying hospital performance to the benchmark level, there were five indicators of acute care and one indicator of subacute care increased by more than 20% in the overall adherence rate. The overall adherence rate of venous thrombosis prophylaxis increased by an average of 17.10% (95% CI: 13.57–20.62%). The proportion of independent patients at discharge increased by more than 20% of two indicators of acute care and 16.98% (95% CI: 14.60–19.36%) of venous thrombosis prophylaxis. The QIs with high increase in both overall adherence and independence rate were thrombolytic therapy, anticoagulant therapy, venous thrombosis prophylaxis. To achieve a 1% increase in the proportion of independent patients at discharge, the adherence rate of the three QIs needs to be increased by less than 2%. When the HPCP of all hospitals were improved to benchmark level, the composite performance increased by 19.86% (95% CI: 15.92–23.81%) and independence rate increased by 35.33% (95% CI: 29.56–41.10%).

Table 3.

Data-driven achievable benchmarks for individual QIs and their predicted quality achievement

Quality indication	Benchmark subset^a	Achievable benchmark, %	IAR (95%CI), %	IIR (95%CI), %
1, Acute treatments
CT or MRI	21,676	97.98	23.25 (19.51, 26.98)	8.18 (6.04, 10.31)
Laboratory tests	22,088	95.34	42.65 (37.01, 48.30)	3.40 (1.36, 5.43)
ECG	21,465	94.03	43.80 (38.26, 49.34)	12.53 (10.46, 14.60)
Thrombolytic therapy	12,818	47.97	23.56 (20.94, 26.17)	22.21 (20.14, 24.28)
Anticoagulant therapy	1520	90.20	31.22 (27.05, 35.39)	23.02 (21.15, 24.89)
Antiplatelet therapy	29,316	99.97	1.48 (1.08, 1.89)	1.99 (0.37, 3.61)
Venous thrombosis prophylaxis	15,871	99.82	17.10 (13.57, 20.62)	16.98 (14.60, 19.36)
2, Subacute treatments
Statins therapy	29,316	99.39	7.65 (5.57, 9.73)	5.46 (3.31, 7.61)
Dysphagia screening	39,510	95.57	39.03 (32.98, 45.08)	7.14 (2.55, 11.74)
3, HPCP	282,192	82.47	19.86 (15.92, 23.81)	35.33 (29.56, 41.10)

IQR: interquartile range; CT: computed tomography; MRI: magnetic resonance imaging; ECG: electrocardiograph; HPCP: hospital process composite performance; IAR: improvement in overall adherence rate; IIR: predicted improvement in independence rate with benchmarking.

Benchmark subset was created by ranking hospitals in descending order by adherence of individual quality indicators and by sequentially pooling eligible patients until the combined size for sequential hospitals approached 10% across all eligible hospitals.

A total of 533,117 patients were included in the sensitivity analysis. The predicted quality achievement with benchmarking for QIs slightly changed, but the QIs with high increase in both overall adherence rate and independence rate did not change (see online supplementary table V).

Discussion

Quality-of-care evaluation comparisons between health care providers might reveal current medical service needs and identify performance gaps in the implementation of guidelines. Setting attainable performance targets can facilitate quality improvement by increasing adherence to evidence-based guidelines. Using a nationwide clinical database of patients with AIS, we have demonstrated the positive associations between adherence rates of QIs and independence (mRS score ≤ 2) rates. We provided achievable performance benchmarks for the nine QIs and HPCP, and we speculated the net gain in quality IIR at discharge if all hospitals were to operate at the benchmark level. The QIs with high increase in both overall adherence rate and independence rate were three acute treatments of AIS, including thrombolytic therapy, anticoagulant therapy, venous thrombosis prophylaxis. The results suggested that quality improvement of stroke care should prioritize the three QIs of acute treatments.

Our study found that the proportion of coexisting diseases was higher in the independent group than in the dependent/dead group and the number of coexisting conditions affected the adherence to QIs, possibly because of differences in patients’ motivation to adhere and perceived care benefit. Better adherence to prescribed therapy is associated with better clinical outcomes. Treatment effectiveness is largely dependent on a patient’s willingness and ability to follow the medical recommendations. Patients with comorbid conditions may be more knowledgeable with the importance of receiving appropriate treatment, but the ability to follow the prescribed therapy decreased as the number of coexisting diseases increased.

Achievable performance benchmarks were based on data derived from assessing actual performance in a predefined, objective, and standardized manner. These benchmarks represented an attainable level of excellence that has already been achieved by a few superior providers, so that their practices might be widely understood and emulated.²⁰ Focusing on the top decile of the eligible population ensured that high-performing providers with low case volumes did not unduly influence the benchmark levels.^8,21 Our study extended previous work in this field by predicting the possible quality improvement of performance benchmarks setting.^22,23 Any quality improvement project or technique must anticipate the extent of quality improvement, in terms of both process of care and patient-relevant outcomes, in a relatively reasonable and cost-effective manner. It is usually impractical and time-consuming to conduct randomized trials to determine the effects of care performance-improvement projects on outcomes.²⁴ Instead, relying on the process–outcome association and using self-control method, we predicted the proportion of independent patients at discharge if all hospitals were to operate at the benchmark level, as they would in more strictly controlled randomized trials.⁸ We observed an increase in the independence rate for all QIs. Although this was exclusively a data-based estimation, it does shed light on the desired effect of ensuring timely and high-quality care for stroke. Additionally, the calculation of predicted quality improvement before the benchmark intervention can provide evidences for the rational selection of QIs for performance improvement and provide additional motivation for promoting the implementation of and compliance with interventions.

It is essential to ensure that healthcare benchmarking achieves its objective. We think two elements should be considered when determining which QIs to prioritize in improving the quality of stroke care: there is enough room for the overall adherence rate to improve (bigger IAR) and it has great impact on outcomes (bigger IIR). Antiplatelet and statins therapy have high starting median adherence rate (98.75% and 92.07%); thus, the ceiling effect may have limited the degree to which any institution could improve their performance. The QIs with larger IAR and larger IIR values were thrombolytic therapy, anticoagulant therapy, venous thrombosis prophylaxis. These QIs are acute treatments of AIS, especially thrombolytic therapy. Research shows that increased quality of care during the early phase of stroke could significantly reduce the risk of disability and mortality.²⁵ Brain is exquisitely sensitive to ischemic injury and most irreversible injury occurs within 4–6 h after stroke onset.²⁶ Placebo-controlled trials have shown a substantial benefit of treatment with rt-PA in AIS patients within 3 h of stroke onset.^27,28 Failure to seek medical attention promptly and the strict therapeutic window for rt-PA delivery are probably contributors to the low adherence rate. Anticoagulant therapy, venous thrombosis prophylaxis should be prescribed within 48 h of admission. In patients with reduced mobility, stroke is strongly associated with an increased risk of DVT. DVT may lead to pulmonary emboli, a frequent cause of avoidable deaths.²⁹ Large room for improvement in both adherence rate and independence rate suggested that many patients still do not receive optimal acute cares, and this in turn is responsible for many avoidable disabilities and deaths. Setting performance targets for the above three indicators could best motivate positive change in both overall adherence rate and independence rate at discharge.

There exist several limitations in this study. Firstly, the roles of indicators in the causal pathway between process care and outcomes can influence IIR. Indicators such as diagnosis and examination indicators don’t have a direct association with outcome. They are part of the causal pathway. This may be one of reasons why the IIR of these indicators were lower than those with level 1 evidences to improve patient outcomes. However, the causal pathway between process care and outcomes is highly complex and the inference of the causality is highly limited by the retrospective cross-sectional nature of our study. Secondly, the 10% target level was somewhat arbitrary. Achievable benchmark setting is flexible, and other users could, if they wished, select a different cutoff percentage, like 15%,²³ 25%.³⁰ Thirdly, a more in-depth covariate analysis of patient and provider characteristics is needed to better understand the process–outcome association. Fourthly, because we did not have access to outpatient follow-up information, we were unable to obtain mRS at a fixed time poststroke, e.g., at 30 or 90 days. Lastly, this is a data-based estimate only, whether improvements in outcomes will follow improvements in performance remains to be seen.

Conclusion

We set achievable benchmarks for a broad spectrum of QIs based on long-term medical records on stroke care practices across China and determined the predicted quality improvement in independence (mRS score ≤ 2) rate at discharge if all hospitals were to operate at the benchmark level. Performance targets for three acute treatments, including thrombolytic therapy, anticoagulant therapy, venous thrombosis prophylaxis, could best motivate improvements in both overall adherence rate and independence rate at discharge. The finding suggests that the above three types of acute treatment should be given priority to improve the quality of AIS care.

Footnotes

Acknowledgements

The authors thank Chang Yin for providing data support in the study.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by National Natural Science Foundation of China (81573255 to Meina Liu).

ORCID iDs

Chao Wang

Meina Liu

Supplemental material

Supplemental material for this article is available online.

References

Murray

CJL

Vos

Lozano

, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012; 380: 2197–2223.

Feigin

Mensah

Murray

CJL

Roth

. Atlas of the global burden of stroke (1990–2013): the GBD 2013 study. Neuroepidemiology 2015; 45: 230.

Strong

Mathers

Leeder

Beaglehole

. Preventing chronic diseases: how many lives can we save? Lancet 2005; 366: 1578–1582.

O’Donnell

Chin

Rangarajan

, et al. Global and regional effects of potentially modifiable risk factors associated with acute stroke in 32 countries (INTERSTROKE): a case-control study. Lancet 2016; 388: 761–775.

Liping

David

Wong

KSL

Yongjun

. Stroke and stroke care in China: huge burden, significant workload, and a national priority. Stroke 2011; 42: 3651–3654.

Romano P, Hussey P and Ritley D. Selecting Quality and Resource Use Measures: A Decision Guide for Community Quality Collaboratives. Washington, DC: Agency for Healthcare Research and Quality; 2010. AHRQ publication 09(10)–007.

Ettorchi-Tardy

Levif

Michel

. Benchmarking: a method for continuous quality improvement in health. Healthc Policy 2012; 7: e101–e119.

Weissman

Allison

Kiefe

, et al. Achievable benchmarks of care: the ABCs of benchmarking. J Eval Clin Pract 1999; 5: 269–281.

Parker

Schwamm

Fonarow

Smith

Reeves

. Stroke quality metrics: systematic reviews of the relationships to patient-centered outcomes and impact of public reporting. Stroke 2012; 43: 155–162.

10.

Annette

Grethe

Hundborg

Svendsen

Johnsen

SRP

. Processes of care and medical complications in patients with stroke. Stroke 2011; 42: 167–172.

11.

Hsieh

Lien

Chen

, et al. Get with the guidelines-stroke performance indicators: surveillance of stroke care in the Taiwan stroke registry: get with the guidelines-stroke in Taiwan. Circulation 2010; 122: 1116–1123.

12.

Fonarow

Reeves

Smith

, et al. Characteristics, performance measures, and in-hospital outcomes of the first one million stroke and transient ischemic attack admissions in get with the guidelines-stroke. Circ Cardiovasc Qual Outc 2010; 3: 291–302.

13.

Kasner

. Clinical interpretation and use of stroke scales. Lancet Neurol 2006; 5: 603–612.

14.

Zhang

Gao

Yang

, et al.

Is elevated SUA associated with a worse outcome in young Chinese patients with acute cerebral ischemic stroke?

BMC Neurol 2010; 10: 82.

15.

Saposnik

Cote

Mamdani

, et al. JURaSSiC: accuracy of clinician vs risk score prediction of ischemic stroke outcomes. Neurology 2013; 81: 448–455.

16.

Kiefe

Weissman

Allison

Farmer

Weaver

Williams

. Identifying achievable benchmarks of care: concepts and methodology. Int J Qual Health Care 1998; 10: 443–437.

17.

Shwartz

Ren

Peköz

Wang

Cohen

Restuccia

. Estimating a composite measure of hospital quality from the Hospital Compare database: differences when using a Bayesian hierarchical latent variable model versus denominator-based weights. Med Care 2008; 46: 778–785.

18.

Peterson

Delong

Masoudi

, et al. ACCF/AHA 2010 position statement on composite measures for healthcare performance assessment: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Performance Measures (Writing Committee to develop a position statement on composite measures). Circulation 2010; 121: 1780–1791.

19.

Anderson

Nichol

Dai

, et al. Association between hospital process composite performance and patient outcomes after in-hospital cardiac arrest care. JAMA Cardiol 2016; 1: 37–45.

20.

Powis

Sutradhar

Gonzalez

, et al. Establishing achievable benchmarks for quality improvement in systemic therapy for early-stage breast cancer. Cancer 2017; 123: 3772–3780.

21.

Wessell

Liszka

Nietert

, et al. Achievable benchmarks of care for primary care quality indicators in a practice-based research network. Am J Med Qual Off J Am Coll Med Qual 2008; 23: 39–46.

22.

Hinchey

Shephard

Tonn

Ruthazer

Selker

Kent

. Benchmarks and determinants of adherence to stroke performance measures. Stroke 2008; 39: 1619–1620.

23.

Hall

Khan

Bayley

, et al. Benchmarks for acute stroke care delivery. Int J Qual Health Care 2013; 25: 710–718.

24.

Palmer

. Process-based measures of quality: the need for detailed clinical data in large health care databases. Ann Intern Med 1997; 127: 733–738.

25.

Ingeman

Pedersen

Hundborg

, et al. Quality of care and mortality among patients with stroke: a nationwide follow-up study. Med Care 2008; 46: 63–69.

26.

Zivin

. Factors determining the therapeutic window for stroke. Neurology 1998; 50: 599–603.

27.

Hacke

Kaste

Bluhmki

, et al. Thrombolysis with alteplase 3 to 4.5 hours after acute ischemic stroke. N Engl J Med 2008; 359: 1317–1329.

28.

National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med 1995; 333: 1581–1587.

29.

Albers

Amarenco

Easton

Sacco

Teal

. Antithrombotic and thrombolytic therapy for ischemic stroke: the Seventh ACCP Conference on Antithrombotic and Thrombolytic Therapy. Chest 2004; 126: 483S–512S.

30.

Muller

Marcon

Sapisochin

, et al. Defining benchmarks in liver transplantation: a multicenter outcome analysis determining best achievable results. Ann Surg 2018; 267: 419–425.