Abstract
Acute coronary syndromes (ACS) are the most common cardiovascular diseases and are associated with a significant risk of mortality and morbidity. The Global Registry of Acute Coronary Events (GRACE) risk score postdischarge is a widely used ACS prediction model for risk of mortality (low, intermediate, and high); however, it has not yet been validated in patients from the Arabian Gulf. This prospective multicenter study (second Gulf Registry of Acute Coronary Events) provides detailed information of the GRACE risk score postdischarge in patients from the Arabian Gulf. Its prognostic utility was validated at 1-year follow-up in over 5000 patients with ACS from 65 hospitals in 6 Arabian Gulf countries (Bahrain, Saudi Arabia, Qatar, Oman, United Arab Emirates, and Yemen). Overall, the goodness of fit (Hosmer and Lemeshow statistic P value = .826), calibration, and discrimination (area under the receiver operating characteristic curve = 0.695; 95% confidence interval: 0.668-0.722) were good. The GRACE risk score postdischarge can be used to stratify 1 year mortality risk in the Arabian Gulf population; it does not require further calibration and has a good discriminatory ability.
Introduction
Nowadays, acute coronary syndromes (ACS) are a highly prevalent group of cardiovascular diseases (CVDs) that include unstable angina, non-ST-segment elevation myocardial infarction (NSTEMI), and ST-segment elevation myocardial infarction (STEMI). Acute coronary syndromes are associated with a significant risk of adverse events, including mortality, morbidity, recurrent ischemia, and bleeding complications. However, the risks of short- and long-term mortality associated with ACS vary hugely according to the condition of the patient. 1 As such, early risk stratification plays an important role as the benefits of more aggressive treatment need to be weighed against the increasing risks of adverse outcomes; thus, it is important to target aggressive treatment only at those patients who are at a higher risk for mortality at the time of admission.
Single markers of risk, such as blood levels of cardiac troponin, are not sufficiently accurate predictors of outcome to be clinically useful. 2 Therefore, various risk stratification methods have been developed which incorporate multiple determinants that have been derived from randomized clinical trials or from registry data, adding more accuracy than is possible through clinical assessment alone. 3 Risk models derived from randomized trials, such as the Thrombolysis in Myocardial Infarction score, 4 have the advantage of being developed from robust scientific data but are based on selected patient cohorts, and their external validity in the wider population of patients with ACS is likely to be limited. In contrast, models derived from registries, such as the Global Registry of Acute Coronary Events (GRACE), 5 –7 are based on analyses of observational data from large unselected populations, often enrolled across many different countries and may provide stronger external validity.
The two main GRACE risk scores now available are those for predicting in-hospital mortality and mortality from discharge to 6 months. The score is a numerical sum of individual values attributed to a several clinical factors known to be relevant to the outcome (ie, mortality) such as heart rate, blood pressure, congestive heart failure, renal impairment, ST-segment elevation on the electrocardiogram, cardiac arrest, and troponin rise.
The discharge to 6-month GRACE risk score has been validated in European populations, 8 and this score has also been shown to be useful to predict up to 2-year postdischarge mortality. 9 However, this score has not yet been validated in patients from the Arabian Gulf, and given that the risk factor profiles and medical management may vary significantly from region to region, the validation of this score for this population is imperative. Therefore, the current study aimed to validate the GRACE risk score after discharge for stratification of long-term (1 year) mortality in patients from the Arabian Gulf.
Methods
Study Design and Population
The data were collected from a prospective multicenter study of the second Gulf Registry of Acute Coronary Events (Gulf RACE-2) that recruited consecutive patients with ACS from 6 Arabian Gulf countries (Bahrain, Saudi Arabia, Qatar, Oman, United Arab Emirates [UAE], and Yemen). Patients were recruited from 65 hospitals between October 2008 and June 2009 with the diagnoses of unstable angina, non-ST-segment elevation acute coronary syndrome (NSTEACS), NSTEMI (included as NSTEACS), and STEMI. Diagnosis of NSTEACS and definitions of data variables were based on the American College of Cardiology clinical data standards. 10 The study received ethical approval from the institutional ethical bodies in all participating countries.
Computation of the GRACE Risk Score Postdischarge
The score was derived from the GRACE registry cohort rules created from analysis of baseline variables in relation to 6-month postdischarge mortality in the original cohort. 5 This GRACE score was analyzed in relation to our 1-year postdischarge mortality data also stratified by STEMI and NSTEMI/NSTEACS as in the international study.
The GRACE risk scores postdischarge to 1 year between patients from different Gulf countries were compared to assess the validity of the score across the region. Among the 7930 patients enrolled in the Gulf RACE-2, the GRACE risk score postdischarge could be calculated in 7049 (88.9%) patients. Of these, 362 were excluded because they had died in hospital since the prognostic score was aimed at those who survived to discharge. Of the remaining 6751 patients, the outcome was unknown at 1 year for 1638. The remaining 5113 patients represented the cohort for the present analysis.
Statistical Analyses
For categorical variables, frequencies and percentages were reported. Differences between groups were analyzed using Pearson χ2 tests. For continuous variables, medians and interquartile ranges were presented, and differences were analyzed using Mann-Whitney test. Given the significant losses to complete follow-up at 1 year (outcome information missing), the analyses were adjusted for the dropouts at 1 year by running an appropriately weighted regression. 11 This was done by first computing the propensity (probability) to remain in the study at 1 year by regressing (using logistic regression) an indicator variable on the baseline explanatory variables. The fitted model gives a predicted probability for each participant that a participant with these characteristics (ie, values on the explanatory variables) would remain at 1 year. These explanatory variables included country of residence, family history of coronary artery disease, ethnic origin, age, and body mass index in addition to the medical history of percutaneous coronary intervention, CVD, and diabetes. Each participant who remained at the 1-year follow-up was then given weight equal to 1/P, where P is their fitted probability of being present at follow-up from the logistic regression. This weighted model–derived mortality rates were used instead of the unweighted (observed) mortality to prevent the introduction of bias if dropouts were associated with the outcome (ie, mortality).
The association between 1-year postdischarge mortality and the GRACE risk score was then assessed using an inverse probability–weighted logistic regression with a robust error variance fitted to the data. This association was also assessed by STEMI versus NSTEACS and as well as for the different countries. Subanalyses by sex and age were also conducted to examine the performance of the GRACE risk score particularly in women and younger patients. When validating logistic regression models, the major concern is how well the predicted probabilities agree with the responses within the independent sample. A goodness-of-fit statistic provides a summary measure of the deviations of individual predicted probabilities from the actual outcomes, and this was examined using the Hosmer and Lemeshow goodness-of-fit statistic (HLS).
The first major source of such deviations is when the new site has a different prevalence of outcomes than the site of model development. This component was assessed as the accuracy of the calibration of the GRACE risk scores and was determined by plotting the predicted versus the actual mortality rates across 3 risk categories defined by cut points of the GRACE score (low, intermediate, and high risk). These cut points were defined as in the derivation study to be 89 to 118 for intermediate-risk NSTEACS and 100 to 127 for intermediate-risk STEMI. Values on either side of the intermediate risk were low (<89 and <100) and high risk (>118 and >127) for NSTEACS and STEMI, respectively. A second major source of such deviations is the ability of the model to discriminate between observations that have positive and negative outcomes, and this was assessed using the area under the receiver operating characteristic (ROC) curve. All analyses were conducted using Stata version 13 (StataCorp, College Station, Texas), and P < .05 was taken to be the cutoff for significance.
Results
Patients’ Baseline Characteristics
Demographic and clinical characteristics stratified by the type of ACS are shown in Table 1 for all participants at baseline. Our population was relatively younger with more male patients and a higher prevalence of diabetes than the GRACE cohort. Of the 7930 patient admitted to hospital, 362 (4.6%) died in hospital, 573 (8.2%) died within 1 month, and 772 (12.6%) died within 1 year of admission (Tables 2 and 3). The GRACE risk score in those who died was significantly higher compared with those who were alive at discharge (P < .001).
Baseline Characteristics of the Gulf RACE-2 Population and the GRACE Risk Score Derivation Cohort.
Abbreviations: CABG, coronary artery bypass grafting; CK creatine kinase; DBP, diastolic blood pressure; ECG, electrocardiogram; GRACE, Global Registry of Acute Coronary Events; Gulf RACE-2, Second Gulf Registry of Acute Coronary Events; HR, heart rate; IQR, interquartile range; MI, myocardial infraction; NSTEACS, non-ST-segment elevation acute coronary syndrome; PAD, peripheral artery disease; PCI, percutaneous coronary intervention; SBP, systolic blood pressure; STEMI, ST-segment elevation myocardial infarction; TIA, transient ischemic attack.
Model-Derived Mortality Rates.a ,b
Abbreviation: GRACE, Global Registry of Acute Coronary Events.
aBoxes highlight outliers.
bInverse probability–weighted rates by group.
cNumbers in each group that were nonmissing for outcome and GRACE scores.
Observed Mortality Rates by GRACE Category.a
aBoxes highlight outliers.
Model Validation
Despite the differences in their clinical presentation characteristics, the GRACE score was suitable to be used in patient from the Arabian Gulf countries. The overall goodness of fit (HLS P = .826), calibration (Figure 1), and discrimination (overall area under the ROC curve = 0.695; 95% confidence interval [CI]: 0.668-0.722) were found to be good. For STEMI, the goodness of fit (HLS P = .818) and discrimination (area under the ROC curve = 0.681; 95% CI: 0.638-0.724) were also found to be good. Similarly, for NSTEACS, goodness of fit (HLS P = .665) and discrimination (area under the ROC curve = 0.707; 95% CI: 0.671-0.742) were found to be good. When individual countries were examined, Yemen performed poorly on both discrimination and calibration, whereas UAE performed poorly on calibration (Figures 2 and 3). The goodness of fit of the GRACE score was good for females (HLS P = .620) and for younger patients (HLS P = .532); discrimination also remained good in both cases (Table 4).
Calibration of the Gulf Registry of Acute Coronary Events (Gulf RACE) mortality at 1-year postdischarge using categories defined by Global Registry of Acute Coronary Events (GRACE) cutoff at low, intermediate, and high predicted risk. The circles represent the observed mortality, and its size is proportional to the number of participants in each category. The line represents the predicted mortality; if the circles fall on the diagonal line, this indicates perfect calibration. The x-axis is the predicted 1-year mortality based on our regression model, and the y-axis is the actual (observed) mortality (weighted).
Calibration of the Gulf Registry of Acute Coronary Events (Gulf RACE) mortality at 1-year postdischarge using categories defined by Global Registry of Acute Coronary Events (GRACE) point cutoff at low, intermediate, and high predicted risk and stratified by country. The circles represent the observed mortality, and its size is proportional to the number of participants in each category. The line represents the predicted mortality; if the circles fall on the diagonal line, this indicates perfect calibration. The outliers (Yemen and the United Arab Emirates) are labeled. The x-axis is the predicted 1-year mortality based on our regression model, and the y-axis is the actual (observed) mortality (weighted).
Discrimination achieved with each country’s data set based on the prediction from an inverse probability–weighted logistic regression on the whole data set. Yemen clearly stands out with poor discriminative ability of the GRACE score. The other countries have more or less similar discriminations.
Model Validation by Sex and Age of the Patients.
Abbreviations: CI, confidence interval; HLS, Hosmer and Lemeshow goodness-of-fit statistic; ROC, receiver operating characteristic.
Discussion
Even though the postdischarge GRACE score has not been validated for our population, hospitals around the Middle East and in the Arabian Gulf countries have been using it routinely to stratify the patients and decide on the level of intervention and treatment. This has been the case, despite the fact that our patients are typically younger and have higher rates of diabetes mellitus. Although the importance of such validation of the GRACE score has been realized for a long time in the region, there were no registry data that had captured long-term mortality outcomes. A previous study had looked at in-hospital mortality and found that this too was validated, although country-specific differences were not explored. 12 The fact that we were able to validate it at 1 year in this study is consistent with the prior finding that it validated well for in-hospital mortality.
Our results suggest that despite overall validation in this population, data from Yemen and UAE differed from the overall cohort as both demonstrated poor calibration. However, discrimination within the Yemeni population was also lost but not so in the UAE population. This is most likely due to the poorer level of medical care in the Yemeni health-care system such that other health-care delivery issues overshadow the prognostic impact of the GRACE score. On the other hand, discrimination was maintained in the UAE cohort and further examination suggests that the GRACE cutoffs used matched those defined by GRACE score tertiles within the UAE and the reason for lack of calibration was a markedly lower postdischarge mortality rate across risk categories within the UAE. This may suggest either a better health system or less risk factors for mortality in the region. The latter seems more likely as when we looked at the distribution of STEMI and NSTEACS within the UAE, the participants with STEMI made up only 7.7% of the high-risk category but 62.5% of the low-risk category. Thus, the trend within the UAE seemed to be that STEMI were mostly low risk and thus may be the reason for markedly lower mortality rates. Given that CVD risk increases with age and it has a different risk profile between females and males, we analyzed the GRACE score among females and the younger Arabian Gulf patients and found it to have a similar performance. Discrimination was slightly lower at ages <56 years, and this may suggest that caution is required in interpreting the score when risk stratifying younger participants. The results in these participants are consistent with the results of our entire cohort as it also had demographic differences with the original GRACE risk derivation cohort. The median age of the patients enrolled in the Gulf RACE-2 was 10 years younger and the proportion of females enrolled was also significantly lower (21.3% vs 33.5% of females) than the original GRACE risk derivation cohort.
Despite the fact that this study analyzed data from the largest multinational prospective registry of ACS from the Arabian Gulf region, we acknowledge some limitations. Probably, the most important limitation was the loss to follow-up at 1 year, a quarter of the target population (1638 of the 6751; 24.3%) were lost. The largest loss was from Oman (32%) and smallest was from UAE (8%). This could have introduced significant bias, had there been loss to follow-up related to the outcome status; however, this was accounted for by using the inverse probability weighting in the analysis. Furthermore, we were reassured that the loss to follow-up did not introduce bias given that the weighted and nonweighted results were similar.
Our study confirms that the GRACE risk score postdischarge has been validated in Arabian Gulf population. It has been shown that the score does not require further calibration for this specific population and has a good discrimination ability to categorize the patients by their risk of mortality; however, generalizability to patients in UAE and Yemen should be performed with caution and reasons for these discrepancies need to be explored in future studies.
Footnotes
Authors’ Note
All authors contributed to (1) substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data, (2) drafting the article or revising it critically for important intellectual content, and (3) final approval of the version to be published. The sponsor had no role in the study design, data collection, data analysis, writing of the report, or submission of the manuscript.
Acknowledgment
The authors thank all the patients, physicians, and hospitals who took part in Gulf RACE-2.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Gulf RACE-2 is a Gulf Heart Association (GHA) project and was financially supported by the GHA, Sanofi-Aventis, and the Deanship of Scientific Research at King Saud University, Riyadh, Saudi Arabia (Research group number: RG-1436-013). L.F.K. is funded by an Endeavour Postgraduate Scholarship (#3781_2014), an Australian National University Higher Degree Scholarship, and a Fondo para la Innovación, Ciencia y Tecnología Scholarship (#095-FINCyT-BDE-2014).