Abstract
Background:
The HERMES-24 score has recently been proposed to estimate the 90-day outcome in patients with anterior circulation large vessel occlusion (LVO) stroke.
Aim:
We aimed to further validate the HERMES-24 score in a real-world setting of LVO patients who had received endovascular stroke treatment (EVT).
Methods:
We used data of consecutive LVO patients treated with EVT in six Austrian stroke centers. The performance of the HERMES-24 score for outcome prediction at 90 days (modified Rankin Scale (mRS)) was investigated in predefined subgroups of anterior and vertebrobasilar LVO and within or beyond 6 h/unknown time windows.
Results:
We analyzed 1571 patients with available 90-day follow-up. The HERMES-24 score was predictive of 90-day mRS ⩽ 2 for the total cohort, those with anterior LVO <6 h, anterior LVO ⩾6 h/unknown, vertebrobasilar LVO < 6 h, and vertebrobasilar LVO ⩾ 6 h/unknown with c-statistics (95% confidence interval (CI)) 0.90 (0.88–0.92), 0.90 (0.88–0.92), 0.89 (0.86–0.92), 0.88 (0.79–0.98) and 0.98 (0.96–1.0), respectively. The HERMES-24 score also yielded excellent outcome prediction for 90-day mRS ⩽ 3, an ordinal mRS, and mortality in all subgroups (c-statistics: 0.83–0.99).
Conclusion:
The HERMES-24 score is highly predictive for 90-day outcome in real-world patients with LVO stroke treated with EVT regardless of LVO localization or symptom onset to treatment time.
Introduction
Endovascular stroke therapy (EVT) with mechanical thrombectomy has revolutionized acute stroke treatment by improving functional outcome and reducing mortality in patients with large vessel occlusion (LVO). 1 Even though recanalization rates for EVT reach up to 90%, prognostication of functional recovery remains challenging. Such knowledge would not only affect therapeutic efforts but may also support medical teams in decisions regarding de-escalation of treatments, goals of care, and communication with patients and their families.
Recently, the HERMES-24 scoring instrument for outcome prediction in patients with LVO stroke has been proposed. 2 This score has been developed using individual patient data from seven randomized clinical trials on stent retriever mechanical devices versus best medical treatment in patients with anterior circulation LVO within 12 h of last known well (almost all were treated within 6 h). Subsequently, the HERMES-24 score has also been validated in anterior circulation LVO patients with small ischemic cores presenting unwitnessed onset including on awakening or witnessed but late after onset using data from six EVT randomized controlled trials. 3
Although the HERMES-24 score showed excellent accuracy for outcome prediction in both studies, it remains unclear so far whether this instrument is also applicable to LVO patients with occlusions of the vertebrobasilar system or—as critically acknowledged by the authors of the two pilot studies2,3—also to a broader patient population from real-world cohorts.
Therefore, we aimed to investigate the predictive ability of the HERMES-24 score in a consecutive cohort of LVO patients who had received EVT for LVO of the anterior or vertebrobasilar circulation at six Austrian stroke centers.
Methods
Patients were included from six EVT centers within the Austrian stroke unit registry, a nationwide quality control registry. Since October 2013, the Austrian Federal Ministry of Health has regulated that all patients undergoing EVT have to be prospectively documented within this registry in currently 12 Austrian thrombectomy centers. The detailed structure and the method of data entry of the Austrian stroke unit registry have been described earlier.4,5
For this study, we used data of all patients ⩾ 18 years who were registered with a diagnosis of acute ischemic stroke receiving EVT of acute LVO between October 1, 2015, and December 2023. LVO comprised occlusions of the M1 and M2 segments of the middle cerebral artery, the intracranial internal carotid artery, the basilar artery, and the V4 segment of the vertebral artery. The HERMES-24 score was calculated as follows: age/10 + National Institutes of Health Stroke Scale (NIHSS) score at 24 h after EVT, which is routinely assessed prospectively in the current registry. Functional outcome was scored at 90 days after stroke using the modified Rankin Scale score (mRS) by neurologists during an outpatient follow-up examination. Outcomes of interest included a good functional outcome defined as an mRS score of ⩽ 2, an mRS score of ⩽ 3, an ordinal mRS score, and mortality.
Statistical analysis
Statistical differences were assessed by the Pearson χ2 test, the Fisher exact test, or the Mann–Whitney U-test, as appropriate. The discriminative value of the HERMES-24 score for investigated outcomes was tested by the area under the receiver operating characteristic curve (AUC; c-statistic). For binary outcomes, discrimination was assessed using the c-statistic with corresponding 95% confidence intervals (CIs) calculated using DeLong’s non-parametric method. For ordinal outcome analysis, the mRS was treated as an ordered categorical variable and analyzed using a proportional odds logistic regression model with the HERMES-24 score as the single predictor (lrm() function, rms package in R). The resulting ordinal c-statistic reflects the probability that, for two randomly selected patients, the patient with the worse observed outcome had a higher predicted risk score. Confidence intervals for the ordinal c-statistic were derived using non-parametric bootstrap resampling. In each bootstrap sample, concordance was recalculated using Harrell’s concordance index for ordinal outcomes, and 95% CIs were obtained from the bootstrap distribution.
Calibration was assessed visually using a bootstrap-corrected calibration plot based on 400 resamples for the primary outcome of a good functional outcome at 90 days (mRS 0–2).
To assess the potential impact of missing 90-day follow-up data, a sensitivity analysis using inverse probability weighting (IPW) was performed. The probability of having available 90-day mRS data was estimated using a logistic regression model including the HERMES-24 score (incorporating age and NIHSS at 24 h), prestroke mRS, Thrombolysis in Cerebral Infarction (TICI) grade, intravenous (IV) thrombolysis, and general anesthesia. Each patient with complete follow-up was subsequently weighted by the inverse of their predicted probability, and the primary analysis was repeated in this IPW-weighted sample. A p-value of < 0.05 was considered statistically significant.
All data were processed using the statistical software R (version 4.4.3).
Standard protocol approvals, registrations, and patient consents
The entry of pseudonymized data into the Austrian Stroke Registry is mandated by a law regulating the quality of healthcare in Austria. Therefore, individual patient consent is not necessary.
Data availability
Data that support the findings of this study are available from the corresponding author upon reasonable request.
Results
The total cohort comprised 2268 consecutive patients with LVO who had received EVT; 1571 of those had available 90-day follow-up information including mRS and were considered for the final analysis. Baseline characteristics for patients with and without available follow-up are shown in Table 1. The median NIHSS score was 16 at baseline and 7 at 24 h after EVT. At 90 days, mRS scores ⩽ 2 and ⩽ 3 were observed in 40.9% and 52.8%, and 26.4% had died.
Baseline characteristics in patients with and without available follow-up information at 90 days after mechanical thrombectomy.
SD: standard deviation; mRS: modified Rankin Scale; NIHSS: National Institutes of Health Stroke Scale; LVO: large vessel occlusion; IV: intravenous; TICI: Thrombolysis in Cerebral Infarction; MT: mechanical thrombectomy; N.A.: not applicable.
The discriminative ability of the HERMES-24 score for the prediction of a functional 90-day outcome in this real-world cohort was excellent for the whole population and for the four predefined subgroups (Table 2). The HERMES-24 score was predictive of 90-day mRS scores ⩽ 2 for the total cohort (n = 1571), anterior LVO < 6 h (n = 879), anterior LVO ⩾ 6 h/unknown (n = 524), vertebrobasilar LVO < 6 h (n = 90), and vertebrobasilar LVO ⩾ 6 h/unknown (n = 78) with c-statistics (95% CI) 0.90 (0.88–0.92), 0.90 (0.88–0.92), 0.89 (0.86–0.92), 0.88 (0.79–0.98), and 0.98 (0.96–1.0), respectively. The HERMES-24 score also yielded excellent outcome prediction for 90-day mRS scores ⩽ 3, ordinal mRS, and mortality at 90 days in all subgroups (c-statistics: 0.83–0.99, Table 2).
Outcome prediction by the HERMES-24 score at 90 days poststroke.
AUC: area under the curve; mRS: modified Rankin scale.
Observed probabilities for outcomes according to the HERMES-24 score points are visualized in Figure 1. Moreover, the HERMES-24 score demonstrated high discriminative performance across different patient subgroups (Figure 2). The bootstrap-corrected calibration plot demonstrated good agreement between predicted and observed probabilities, with a mean absolute calibration error of 0.012.
Observed probabilities for outcomes according to the HERMES-24 score.
Discriminative performance of the HERMES-24 score by subgroups.
The IPW-adjusted analysis yielded results consistent with the primary complete-case analysis and did not alter the effect estimates. In particular, the HERMES-24 regression coefficient for predicting 90-day mRS ⩽ 2 remained essentially unchanged after weighting (complete-case estimate: −0.031 vs. IPW-adjusted: −0.031; both p < 0.001).
Discussion
In this multicenter real-world cohort, we confirm the excellent prognostic value of the easily applicable HERMES-24 score for predicting functional neurological outcome and mortality after EVT of acute intracranial LVO of the anterior cerebral circulation in both early and extended/unknown time windows. As a novel finding, we were able to show that the HERMES-24 score is also highly predictive for 90-day outcomes in patients who had received EVT for vertebrobasilar LVO, again irrespective of the time window. This is important as both recent prior studies deriving and validating the HERMES-24 score in patients with anterior LVO used randomized clinical trial populations with restrictive inclusion and exclusion criteria/known differences compared to real-world patients.2,3,6
Accordingly, in comparison to the EVT arms in the HERMES and AURORA trial populations, patients analyzed in this study were a few years older and had nearly equal stroke severity at admission but worse 90-day outcomes, most likely explained by the real-world clinical setting and the inclusion of patients with vertebrobasilar LVO who, as is known, have worse outcomes than patients with anterior LVO. 7
As a limitation, we have to acknowledge that some baseline information such as the ASPECTS score was not uniformly available and could not be analyzed. Most importantly, only around 70% of the originally investigated study patients had available 90-day follow-up data, and only those could be considered for the final analysis. Overall, there were slight differences in baseline characteristics between patients with and without available follow-up. However, an IPW sensitivity analysis explicitly accounting for these differences yielded results consistent with the complete-case analysis, suggesting that missing outcome data did not substantially bias our findings. Moreover, we included patients from both academic (n = 2) and non-academic (n = 4) stroke centers with a good geographical spread over the country, again supporting the inclusion of a representative population. Finally, the numbers of vertebrobasilar LVO patients were low, and results should therefore be interpreted cautiously and regarded as preliminary rather than definitive validation. Further data from larger international multicenter studies on the performance of the HERMES-24 score in this clinically important subgroup are warranted to validate our findings.
Footnotes
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
