IER-START nomogram for prediction of three-month unfavorable outcome after thrombectomy for stroke

Abstract

Background

The applicability of the current models for predicting functional outcome after thrombectomy in strokes with large vessel occlusion (LVO) is affected by a moderate predictive performance.

Aims

We aimed to develop and validate a nomogram with pre- and post-treatment factors for prediction of the probability of unfavorable outcome in patients with anterior and posterior LVO who received bridging therapy or direct thrombectomy <6 h of stroke onset.

Methods

We conducted a cohort study on patients data collected prospectively in the Italian Endovascular Registry (IER). Unfavorable outcome was defined as three-month modified Rankin Scale (mRS) score 3–6. Six predictors, including NIH Stroke Scale (NIHSS) score, age, pre-stroke mRS score, bridging therapy or direct thrombectomy, grade of recanalization according to the thrombolysis in cerebral ischemia (TICI) grading system, and onset-to-end procedure time were identified a priori by three stroke experts. To generate the IER-START, the pre-established predictors were entered into a logistic regression model. The discriminative performance of the model was assessed by using the area under the receiver operating characteristic curve (AUC-ROC).

Results

A total of 1802 patients with complete data for generating the IER-START was randomly dichotomized into training (n = 1219) and test (n = 583) sets. The AUC-ROC of IER-START was 0.838 (95% confidence interval [CI]): 0.816–0.869) in the training set, and 0.820 (95% CI: 0.786–0.854) in the test set.

Conclusions

The IER-START nomogram is the first prognostic model developed and validated in the largest population of stroke patients currently candidates to thrombectomy which reliably calculates the probability of three-month unfavorable outcome.

Keywords

Acute stroke therapy thrombectomy endovascular treatment thrombolysis outcome nomogram

Introduction

Endovascular thrombectomy for ischemic stroke with large vessel occlusion (LVO) substantially reduces disability, with five randomized clinical trials (RCTs) leading to guideline changes worldwide.^1–5 Intravenous thrombolysis (IVT) plus mechanical thrombectomy, or direct thrombectomy in patients with contraindications for IVT, are recommended within 6 h of stroke onset.⁶ Early and successful recanalization of the occluded vessel is critical to achieve the best possible functional outcome across all RCTs.^1–5

In the last few years, several scores based on pre-treatment clinical and radiological variables have been applied in anterior circulation strokes with the aim of standardizing patient selection for thrombectomy;^7–9 however, their applicability for predicting functional outcome is affected by a moderate predictive performance. Recently, the START (National Institutes of Health STroke Scale [NIHSS] score, Age, pre-stroke modified Rankin Scale [mRS] score, and symptom onset-to-needle Time) has been the first nomogram developed and validated in a large Italian cohort which reliably calculates the probability of three-month unfavorable outcome in stroke patients who received IVT alone.¹⁰ The discriminative performance of the model was discrete in a small group of patients who also underwent thrombectomy after IVT.

Aims

The aim of this study was to develop and validate the Italian Endovascular Registry (IER)-START nomogram including NIHSS score, age, pre-stroke mRS score, bridging therapy or direct mechanical thrombectomy, grade and time of recanalization for individualized prediction of the probability of three-month unfavorable outcome in stroke patients with LVO of anterior or posterior circulation who received mechanical thrombectomy within 6 h of symptom onset.

Methods

Study design and participants

We conducted a cohort study on patients data collected prospectively in the IER. The Italian Registry of Endovascular Stroke Treatment in Acute Stroke is a multicenter, observational internet-based registry (Supplemental Table 1). All acute ischemic stroke patients with LVO who received endovascular procedures between January 2011 and December 2016 in the participating centers were included in the present study. All participating centers were required to accept the rules of the IER, including consecutive registration of all stroke patients receiving endovascular procedures, irrespective of whether treatment was according to guidelines. Our analysis was conducted according to the STROBE criteria for observational studies.¹¹

Data collection

The collected data are provided in the Supplementary Material.

Criteria for development of the model

Similarly to the START nomogram,¹⁰ we choose a priori to develop the nomogram model with a limited number of variables that are readily available for patients who undergo mechanical thrombectomy. Among the clinical and radiological factors, which influence outcome after mechanical thrombectomy, six strong predictors available in the IER were identified by three neurologists with clinical expertise in stroke management. Baseline NIHSS score, age, and pre-stroke mRS score were three of the four putative pre-treatment clinical predictors of the START nomogram¹⁰ that entered into the model. The fourth predictor of the START nomogram (i.e. symptom onset-to-needle time for IVT time) was not available in the IER. Since a recent meta-analysis on outcome of stroke patients undergoing mechanical thrombectomy with or without IVT¹² supports the current guidelines of offering IVT to eligible patients even if they are being considered for mechanical thrombectomy,¹³ direct mechanical thrombectomy (bridging of mechanical thrombectomy with IVT as reference) entered into the model. Moreover, the benefit of mechanical thrombectomy decreases as onset-to-reperfusion time increases and recanalization rate decreases across all RCTs,^1–5 symptom onset-to-end procedure time (as non-categorical predictor) and grade of reperfusion according to the TICI grading system (TICI 3 as reference)¹⁴ were two putative strong post-treatment radiological predictors identified to enter the model. The ASPECT and collateral circulation scores^15,16 were disregarded as the model was developed and validated by including also a cohort of stroke patients with posterior LVO.

Inclusion and exclusion criteria

We only included patients with complete data of all variables present in the nomogram and three-month mRS score. Age ≥ 18 years was selected in agreement with the current guidelines.⁶ In addition, we selected the range 0–2 for pre-stroke mRS score as patients with pre-stroke mRS > 2 have three-month unfavorable outcome.

Patients who received intra-arterial (IA) fibrinolysis or started mechanical thrombectomy after 360 min of stroke onset were excluded from the analysis in agreement with the current guidelines.⁶

Outcome

The outcome measure was unfavorable functional outcome defined as mRS score 3–6 at three months.

Statistical analysis

The IER cohort was randomly dichotomized into training and test sets by the statistical software STATA 13.0.1 (StataCorp, College Station, Texas, USA): 2/3 of the cohort was used to develop the prediction model, while the remaining 1/3 to perform the validation of the model.

Differences between the cohorts were explored using the Mann–Whitney U-test for continuous variables. Differences between proportions were assessed by Fisher’s exact test or χ² test, when appropriate. Continuous variables were reported as median and interquartile range (IQR) values. Proportions were calculated for categorical variables, dividing the number of events by the total number excluding missing/unknown cases.

To generate the nomogram, the pre-established predictors entered a logistic regression model. Collinearity of combinations of variables in the training set was evaluated by the variation inflation factors (VIF, < 2 being considered non-significant) and condition index (<30 being considered non-significant). Regression coefficients with standard error and odds ratios (OR) with two-sided 95% confidence intervals (CI) for each of the variables included in the model were finally calculated.

Discrimination of the nomogram was assessed by calculation of the area under the receiver operating characteristic curve (AUC-ROC). Calibration of the risk prediction model was assessed in the test cohort by the plot comparing the observed probability of unfavorable outcome according to the total score of the nomogram against the predicted probability based on the nomogram and by using the Hosmer–Lemeshow test that assesses whether or not the observed event rates matched the expected rates in subgroups of patients.

Additional analyses are provided in the Supplementary Material.

Standard protocol approvals, registrations, and patient consents

Need for ethical approval or patient consent for participation in the Italian Registry of Endovascular Treatment in Acute Stroke varied among participating hospitals. Ethical approval and patient consent were obtained when required.

Data availability statement

Anonymized data will be shared by request from any qualified investigator.

Results

Among the 3714 patients registered in the IER cohort by 45 centers (Supplemental Table 2), 1802 patients with complete data for generating the nomogram were included in the study. Flow diagram of patient inclusion and exclusion is provided in Figure 1. The clinical characteristics of the patients included and excluded are provided in Supplemental Table 3.

Figure 1.

Flow diagram of included and excluded patients.

The clinical characteristics of the patients in the training (n = 1219) and test (n = 583) cohorts are provided in the Supplemental Table 4. The proportion of patients with three-month unfavorable outcome was 52% in the training cohort and 52.7% in the test cohort.

To generate the IER-START nomogram for prediction of the probability of unfavorable functional outcome in the training cohort, six pre-established predictors were entered into a logistic regression model: age (OR: 1.052; CI: 1.040–1.064; p < 0.001), pre-stroke mRS score (OR: 2.341; 95% CI: 1.722–3.182; p < 0.001), baseline NIHSS score (OR: 1.164; 95% CI: 1.132–1.196; p < 0.001), onset-to-end procedure time (OR: 1.004; 95% CI: 1.002–1.006; p < 0.001), bridging therapy (reference); direct mechanical thrombectomy (OR: 1.406; 95% CI: 1.059–1.868; p = 0.018), TICI 3 (reference), TICI 2b (OR: 1.600; 95% CI: 1.106–2.314; p = 0.013), TICI 2a (OR: 4.194; 95% CI: 2.701–6.513; p < 0.001), TICI 1 (OR: 6.137; 95% CI: 2.538–14.828; p < 0.001), and TICI 0 (OR: 15.313; 95% CI: 7.385–31.752; p < 0.001) (Supplemental Table 5). No significant statistical collinearity was observed for any of the six pre-established variables. The nomogram was created by assigning a preliminary score to each of the six predictors with points ranging between 0 and 10, which was then summed to generate a total score, finally converted into an individual risk of unfavorable outcome after thrombectomy.

The IER-START nomogram is shown in Figure 2 taking into account the approximation of all the variables that are graphed without decimals. Paradigmatic examples of the application of the IER-START nomogram are provided in Figure 3. Case no. 1, a 72-year-old patient with pre-stroke mRS score of 0 and baseline NIHSS score of 21 ends procedure of mechanical thrombectomy (bridging) 159 min after symptoms onset achieving TICI grade 3. His total score is therefore 5.6+0+5+0+1+0 = 11.6 points, and his probability of having unfavorable outcome is around 32%. Case no. 2, a 72-year-old patient with pre-stroke mRS score of 0 and baseline NIHSS score of 21 ends procedure of mechanical thrombectomy (bridging) 631 min after symptoms onset achieving TICI grade 0. His total score is therefore 5.6+0+5+0+3.8+4.3 = 18.7 points, and his probability of having unfavorable outcome is around 97%.

Figure 2.

IER-START nomogram for predicting the probability of three-month unfavorable outcome.

Figure 3.

Example of using the IER-START nomogram: two paradigmatic cases. Mark an individual’s onset-to-end procedure time on the “onset-to-end procedure time (minutes)” axis, and draw a vertical line to the “Score” axis to determine the number of points the patient receives for his/her onset-to-end procedure time. Repeat this process for the “age (years),” “baseline NIHSS score (points),” “pre-stroke mRS score (points),” “TICI (0, 1, 2 a, 2 b, and 3),” and “mechanical thrombectomy (direct or bridging).” Add the number of points from each predictor. Mark this sum on the “Total score” axis, and draw a vertical line down to meet the “Probability of 3-month unfavorable outcome” axis, to find the patient’ probability of having unfavorable outcome.

The AUC-ROC value of the IER-START nomogram was 0.838 (95% CI: 0.816–0.860) in the training cohort. The model was internally validated using 5000 bootstrap samples to calculate the discrimination with accuracy of 0.820 (95% CI: 0.787–0.854). The model was validated in the test cohort with AUC-ROC value of 0.820 (95% CI: 0.786–0.854). Figure 4 displays a calibration plot, comparing the predicted proportion of patients who developed three-month unfavorable outcome per nomogram with the proportions observed according to IER-START total score point in the test set. The Hosmer–Lemeshow goodness-of-fit test comparing predicted and observed rates of unfavorable outcome showed good calibration of the IER-START total score (7.516; p = 0.482).

Figure 4.

IER-START nomogram calibration plot. Calibration plot displaying the observed proportion of patients who developed three-month unfavorable outcome in the test cohort (dots) with 95% confidence intervals (vertical lines) against IER-START total score values; the dashed curve shows the predicted values based on the IER-START nomogram.

The AUC-ROC values were calculated in the subgroups of patients included in the entire cohort with complete data for generating the nomogram (Supplemental Table 6).

Discussion

We present here the IER-START nomogram as a reliable tool predicting the probability of three-month unfavorable outcome for individual stroke patients receiving mechanical thrombectomy with or without IVT pre-treatment. The discriminative performance of the nomogram was good in the training and test cohorts.

In contrast to previous scores based on clinical and radiological pre-treatment variables for standardizing patient selection,^7–9 we developed a nomogram including pre- and post-treatment variables for predicting the probability of unfavorable outcome in stroke patients currently candidates for thrombectomy. We included a final IER cohort of patients aged ≥ 18 years undergoing thrombectomy within 6 h of symptom onset for acute ischemic stroke due to anterior and posterior LVO; to date, these patients represent the largest population of strokes currently candidates for thrombectomy.¹³ The AUC-ROC values were good across different subgroups of patients identified according to several variables including sex, pre-stroke disability (i.e. mRS score ≤ 1 and > 1), neurological (i.e. NIHSS score ≥ 6 and < 6) and radiological severity (i.e. ASPECT score ≥ 6 and < 6), occlusion site, and grade of collateral circulation. Our model cannot be applied to patients eligible to thrombectomy after 6 h of symptom onset. However, this a therapeutic option for only a few patients who have LVO in the anterior circulation and comply with strict selection criteria according to DEFUSE 3 or DAWN.^17,18

An accurate estimate of the likelihood of treatment success or failure for each acute ischemic stroke patients may provide important information to clinicians when discussing prognosis with patients and their families. By using the combination of six predictors easily available before and at the end of endovascular procedure, the IER-START nomogram may be a new and reliable graphical calculation instrument for providing indications on more or less intensive patient management after revascularization treatments. Baseline NIHSS score is the strongest predictor of unfavorable outcome, while age and pre-stroke mRS score are non-modifiable variables of the individual patient. A very recent study showed that NIHSS score at the end of endovascular procedure conferred better discrimination power for predicting the three-month functional outcome compared with baseline NIHSS score.¹⁹ However, post-treatment NIHSS score is not easily accessible because of general anesthesia or deep sedation during the procedure and logistical reasons. Also, it is a variable strongly dependent by the final result of mechanical thrombectomy. Grade of reperfusion (i.e. TICI system) and symptom onset-to-end procedure time are predictive markers which measure the final result of mechanical thrombectomy; they are related to the combination of several variables such as symptom onset-to-needle time for IVT, symptom onset-to-groin puncture time, occlusion pattern, grade of collateral circulation, thrombus characteristics, stroke etiology, type of procedure, first pass effect, and expertise of the interventional radiologist. Furthermore, pre-treatment with IVT offers an advantage on outcome, albeit modest, compared with direct thrombectomy.

In recent years, several prognostic models have been applied to anterior LVO strokes with the aim of standardizing patient selection for thrombectomy; however, their applicability for predicting functional outcome is affected by a moderate predictive performance. The THRIVE score included age, NIHSS score, and medical co-morbidities (i.e. hypertension, diabetes mellitus, and atrial fibrillation) but did not account for CT findings.⁷ The HIAT2 score additionally included ASPECT score and other variables (i.e. age, glucose, and NIHSS score) as grouped instead of continuous variables probably reducing its predictive accuracy.⁸ The PRE (Pittsburg Response to Endovascular therapy) score included age, NIHSS score, and ASPECT score as continuous variables, and improved predictive accuracy compared to THRIVE and HIAT2 scores in a first validation cohort of stroke patients with anterior LVO.⁹ In the external validation cohort of stroke patients enrolled in TREVO2, successful reperfusion resulted in clear benefit in patients with PRE scores 0–24 and 25–49, but not in those with PRE ≥ 50.²⁰ Certainly, the outcome remains strongly linked to the grade of reperfusion, but it needs to be achieved quickly. The lack of the final result of the endovascular procedure may explain the possible ceiling of predictive accuracy among previous prognostic scores based on pre-treatment predictors. Rather than selecting the patients before the procedure, the IER-START nomogram could help the clinician to estimate the probability of unfavorable outcome after thrombectomy, especially in prognostically uncertain circumstances such as when the treatment achieves a rapid and complete recanalization in severe strokes or when the vessel remains occluded in mild strokes in young or elderly patients with or without pre-stroke disability.

Our study has some limitations. First, it is based on a retrospective analysis of prospectively collected data. Despite our belief that the patient data in the entire Italian cohort are representative of a variety of demographics and stroke center types, for the risk score to be suitable in daily clinical practice, an external validation in a completely different cohort is warranted. Second, the number of missing data for generating the nomogram and three-month follow-up of the mRS score might have influenced the final outcome. Moreover, the AUC-ROC values calculated in the subgroups of patients might have been influenced by the number of missing data of each variable that identified the subgroup. However, to the best of our knowledge, our sets are among the largest cohorts explored to develop a model for prediction of the outcome after thrombectomy.

Nonetheless, this study brings new insights to stroke clinicians as it provides a new and improved model of predicting the probability of three-month unfavorable outcome in stroke patients undergoing thrombectomy, supporting clinical decision in this setting. The IER-START nomogram could be applied to the largest population with stroke patients currently candidates to thrombectomy. Our model may be easily and quickly applicable in the clinical setting if used on a computer or a handheld device with the specific software.

Supplemental Material

Supplemental material for IER-START nomogram for prediction of three-month unfavorable outcome after thrombectomy for stroke

Supplemental Material for IER-START nomogram for prediction of three-month unfavorable outcome after thrombectomy for stroke by Manuel Cappellari, Salvatore Mangiafico, Valentina Saia, Giovanni Pracucci, Sergio Nappini, Patrizia Nencini, Daniel Konda, Fabrizio Sallustio, Stefano Vallone, Andrea Zini, Sandra Bracco, Rossana Tassi, Mauro Bergui, Paolo Cerrato, Antonio Pitrone, Francesco Grillo, Andrea Saletti, Alessandro De Vito, Roberto Gasparotti, Mauro Magoni, Edoardo Puglielli, Alfonsina Casalena, Francesco Causin, Claudio Baracchini, Lucio Castellan, Laura Malfatto, Roberto Menozzi, Umberto Scoditti, Chiara Comelli, Enrica Duc, Alessio Comai, Enrica Franchini, Mirco Cosottini, Michelangelo Mancuso, Simone Peschillo, Manuela De Michele, Andrea Giorgianni, Maria Luisa Delodovici, Elvis Lafe, Maria F Denaro, Nicola Burdi, Saverio Internò, Nicola Cavasin, Adriana Critelli, Luigi Chiumarulo, Marco Petruzzellis, Marco Doddi, Antonio Carolei, William Auteri, Alfredo Petrone, Riccardo Padolecchia, Tiziana Tassinari, Marco Pavia, Paolo Invernizzi, Gianni Turcato, Stefano Forlivesi, Elisa Francesca Maria Ciceri, Bruno Bonetti, Domenico Inzitari and Danilo Toni in International Journal of Stroke

Footnotes

Acknowledgements

We thank all IER collaborators (see Supplementary Material) for data collection. We also thank all patients who participated in IER.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Manuel Cappellari received consulting fees from Boehringer-Ingelheim. Salvatore Mangiafico received consulting fees from Johnson & Johnson. Andrea Zini received speaker fees and consulting fees from Boehringer-Ingelheim and Medtronic, and serves as advisory board from Boehringer-Ingelheim. Domenico Inzitari received research grants from Shire and speaker honoraria from Shire Italia. Danilo Toni received speaker fees and serves as advisory board from Boehringer-Ingelheim Bayer, Pfizer-BMS, Daiichi Sankyo. The other authors report no disclosures.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project ‘‘Registro Nazionale Trattamento Ictus Acuto’’ (RFPS-2006-1-336562) was funded by grants from the Italian Ministry of Health within the framework of 2006 Finalized Research Programs (D.Lgs.n.502/1992).

References

Berkhemer

Fransen

Beumer

, et al. A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med 2015; 372: 11–20.

Goyal

Demchuk

Menon

, et al. Randomized assessment of rapid endovascular treatment of ischemic stroke. N Engl J Med 2015; 372: 1019–1030.

Saver

Goyal

Bonafe

, et al. Stent-retriever thrombectomy after intravenous t-PA vs. t-PA alone in stroke. N Engl J Med 2015; 372: 2285–2295.

Campbell

Mitchell

Kleinig

, et al. Endovascular therapy for ischemic stroke with perfusion-imaging selection. N Engl J Med 2015; 372: 1009–1018.

Jovin

Chamorro

Cobo

, et al. Thrombectomy within 8 hours after symptom onset in ischemic stroke. N Engl J Med 2015; 372: 2296–2306.

Toni

Mangiafico

Agostoni

, et al. Intravenous thrombolysis and intra-arterial interventions in acute ischemic stroke: Italian Stroke Organisation (ISO)-SPREAD guidelines. Int J Stroke 2015; 10: 1119–1129.

Flint

Xiang

Gupta

, et al. THRIVE score predicts outcomes with a third-generation endovascular stroke treatment device in the TREVO-2 trial. Stroke 2013; 44: 3370–3375.

Sarraj

Albright

Barreto

, et al. Optimizing prediction scores for poor outcome after intra-arterial therapy in anterior circulation acute ischemic stroke. Stroke 2013; 44: 3324–3330.

Rangaraju

Aghaebrahim

Streib

, et al. Pittsburgh response to endovascular therapy (PRE) score: optimizing patient selection for endovascular therapy for large vessel occlusion strokes. J Neurointerv Surg 2015; 7: 783–788.

10.

Cappellari

Turcato

Forlivesi

, et al. The START nomogram for individualized prediction of the probability of unfavorable outcome after intravenous thrombolysis for stroke. Int J Stroke 2018; 13: 700–706.

11.

von Elm

Altman

Egger

Pocock

Gotzsche

Vandenbroucke

. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 2007; 370: 1453–1457.

12.

Mistry

Nakawah

, et al. Mechanical thrombectomy outcomes with and without intravenous thrombolysis in stroke patients: a meta-analysis. Stroke 2017; 48: 2450–2456.

13.

Powers

Rabinstein

Ackerson

, et al. 2018 guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2018; 49: e46–e110.

14.

Higashida

Furlan

Roberts

, et al. Trial design and reporting standards for intra-arterial cerebral thrombolysis for acute ischemic stroke. Stroke 2003; 34: e109–e137.

15.

Pexman

Barber

Hill

, et al. Use of the Alberta Stroke Program Early CT Score (ASPECTS) for assessing CT scans in patients with acute stroke. AJNR Am J Neuroradiol 2001; 22: 1534–1542.

16.

Mangiafico

Saia

Nencini

, et al. Effect of the interaction between recanalization and collateral circulation on functional outcome in acute ischaemic stroke. Interv Neuroradiol 2014; 20: 704–714.

17.

Albers

Marks

Kemp

, et al. Thrombectomy for stroke at 6 to 16 hours with selection by perfusion imaging. N Engl J Med 2018; 378: 708–718.

18.

Nogueira

Jadhav

Haussen

, et al. Thrombectomy 6 to 24 hours after stroke with a mismatch between deficit and infarct. N Engl J Med 2018; 378: 11–21.

19.

Jeong

Kim

Choi

, et al. Posttreatment national institutes of health stroke scale is superior to the initial score or thrombolysis in cerebral ischemia for 3-month outcome. Stroke 2018; 49: 938–944.

20.

Ali Raza

Xiang

Jovin

, et al. Pittsburgh response to endovascular therapy score as a pre-treatment prognostic tool: external validation in Trevo2. Int J Stroke 2017; 12: 494–501.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.33 MB