Abstract
Background
The ankle brachial index (ABI) is related to risk of cardiovascular events independent of the Framingham risk score (FRS). The aim of this study was to develop and evaluate a risk model for cardiovascular events incorporating the ABI and FRS.
Design
An analysis of participant data from 18 cohorts in which 24,375 men and 20,377 women free of coronary heart disease had ABI measured and were followed up for events.
Methods
Subjects were divided into a development and internal validation dataset and an external validation dataset. Two models, comprising FRS and FRS + ABI, were fitted for the primary outcome of major coronary events.
Results
In predicting events in the external validation dataset, C-index for the FRS was 0.672 (95% CI 0.599 to 0.737) in men and 0.578 (95% CI 0.492 to 0.661) in women. The FRS + ABI led to a small increase in C-index in men to 0.685 (95% CI 0.612 to 0.749) and large increase in women to 0.690 (95% CI 0.605 to 0.764) with net reclassification improvement (NRI) of 4.3% (95% CI 0.0 to 7.6%, p = 0.050) and 9.6% (95% CI 6.1 to 16.4%, p < 0.001), respectively. Restricting the FRS + ABI model to those with FRS intermediate 10-year risk of 10 to 19% resulted in higher NRI of 15.9% (95% CI 6.1 to 20.6%, p < 0.001) in men and 23.3% (95% CI 13.8 to 62.5%, p = 0.002) in women. However, incorporating ABI in an improved newly fitted risk factor model had a nonsignificant effect: NRI 2.0% (95% CI 2.3 to 4.2%, p = 0.567) in men and 1.1% (95% CI 1.9 to 4.0%, p = 0.483) in women.
Conclusions
An ABI risk model may improve prediction especially in individuals at intermediate risk and when performance of the base risk factor model is modest.
Introduction
The accurate prediction of major cardiovascular events in individuals without vascular disease is an important public health goal, allowing targeting of preventive measures to those at increased risk. The Framingham risk score (FRS), which combines classic cardiovascular risk factors such as cigarette smoking and total cholesterol, is the most well-known method of risk prediction. However, the FRS has limited accuracy, 1 and attempts have been made to improve prediction by incorporating additional risk factors, novel biomarkers, and measures of subclinical atherosclerosis. 2
The ankle brachial index (ABI), which is the ratio of ankle:arm systolic pressure, is a measure of atherosclerosis in symptomatic and asymptomatic individuals. Originally developed as a diagnostic tool for leg artery disease, the ABI is also an indicator of generalized atherosclerosis and of future cardiovascular events, independent of established risk factors. 3 Since the ABI can be measured using only a handheld Doppler probe and sphygmomanometer in many healthcare settings, the possibility of using the ABI to enhance risk prediction is of interest.
The ABI Collaboration was formed of investigators performing longitudinal studies of the ABI and incident cardiovascular events. In a meta-analysis comprising 48,294 subjects, a low ABI (≤0.90) compared to a normal ABI (1.11–1.40) was related to a 2–3-fold increase in both 10-year major coronary events and cardiovascular mortality independent of the FRS. 3 Furthermore, in cardiovascular risk stratification using the FRS, subsequent inclusion of the ABI resulted in reclassification of risk (low, intermediate, high) in 1 in 5 men and 1 in 3 women. However, the impact of reclassification in improving risk prediction was not assessed.
In this present study, our aim was to develop an ABI prediction model incorporating FRS and ABI into a single equation. The ABI model was then evaluated in a different population and the effect of reclassification of 10-year risk assessed. Since risk prediction varies according to gender and ethnic group, 4 the model was developed separately in men and women and only in whites who comprised the largest ethnic group in the ABI Collaboration studies.
Methods
Study design
The study design was an analysis of individual participant data from 18 prospective cohort studies. Risk prediction models were fitted in a development dataset and evaluated in internal and external validation datasets.
Inclusion and exclusion of studies
The study was based on 20 cohort datasets in the ABI Collaboration. The literature search and collation of data from 16 of these studies5–20 were described previously. 3 Since then, experts informed us of four further studies,21–24 from which corresponding data were acquired. Two studies included only non-White ethnic groups7,8 and were excluded, leaving 18 studies in the final analysis. Individual participant data were extracted and analysed using version 9.2 of SAS System for Windows (2002–2008; SAS Institute, Cary, NC, USA).
Subjects, variables, and missing data
Subjects with nonvalid ABI, with prevalent coronary heart disease as defined in each study at baseline, and without follow up for vital status were excluded. Framingham covariates extracted were age, gender, systolic and diastolic blood pressures, total and high-density lipoprotein cholesterol, and smoking and diabetes indicators. ABI, study location, and ethnic group were obtained and only subjects included who were classified as ‘white’ using individual study classifications. Analyses were not performed on non-White ethnic groups because of small sample sizes. The numbers of men and women in the final analysis were 24,375 and 20,377, respectively. Only 7.4% of data in men and 2.9% in women were missing, predominantly for total and high-density lipoprotein cholesterol. Imputation was performed separately by gender using the SAS procedure PROC MI with the MCMC full-data imputation method. FRS was calculated for all individuals using the methods of Wilson et al. 25 In keeping with our previous investigation, 3 ABI was categorized into four groups: ≤0.90, 0.91–1.10, 1.11–1.40, >1.40.
Development, internal validation, and external validation datasets
Participants and characteristics of studies in the development/internal validation and external validation datasets for men and women
Subjects in each of the development/internal validation studies were allocated randomly to either the development or internal validation datasets.
, not available.
Model fitting
Two models were fitted each for the primary outcome of major coronary events (myocardial infarction or death due to coronary heart disease) and the secondary outcome of cardiovascular mortality (death due to coronary heart disease or stroke) using Cox’s proportional hazards model, as follows: model 1: Framingham risk score 25 fitted as a continuous variable (FRS); model 2: as per model 1 with addition of ABI group (FRS + ABI).
These models were first fitted for men and women separately in the development datasets. Details of the fitted FRS + ABI for major coronary events are shown in Supplementary Table 1 and measures of model performance26–28 in Supplementary Table 2 (available online).
For comparison with the FRS, an additional risk factor model was fitted in the development dataset in which, instead of using the overall single FRS value, each individual Framingham risk covariate, such as smoking, was used separately. And then a further model was fitted with addition of ABI group (Supplementary Table 1). Assumptions for Cox’s analyses were checked formally.
Outcome measures
To quantify the effect of inclusion of the ABI in risk prediction models, the C-index and net reclassification improvement (NRI) were derived.
The C-index is analogous to C-statistic, which is the area under the receiver operating characteristic curve and is a measure of the model’s discrimination of events and non-events. The C-index is for survival data, being the fraction of occasions where the predictor score correctly predicts the earlier event for pairs of individuals in the sample. This was achieved using the methods of Harrell et al. 29 and Pencina and D’Agostino 30 adopting the simplified method for confidence intervals.
For the calculation of the NRI, reclassification tables comparing predicted risk categories for models with and without ABI were constructed for the external validation dataset. Ten-year risk categories of <10%, 10–19%, and ≥20% were selected for major coronary events and <2%, 2–4%, and ≥5% for cardiovascular mortality. NRIs were calculated taking account of censored data. 31 Confidence intervals and p-values were derived using methods for the standard NRI.
For major coronary events, analysis was repeated with a wider intermediate group of 5–19%, akin to some recent studies of risk prediction.32,33
Results
Predicting events in the whole study population
C-indices for Framingham risk score predictions of major coronary events and cardiovascular mortality in men and women with and without the ankle brachial index for the development and internal and external validation datasets
Values are C-index (95% CI)
The number of subjects in major coronary events data sets are lower than in cardiovascular mortality datasets because of nonfatal events not ascertained in four studies in men (Health in Men, Mr Os, Hoorn, INCHIANTI) and three studies in women (Hoorn, InCHIANTI, Study of Osteoporotic Fractures)
ABI, ankle brachial index; cardiovascular mortality, death due to coronary heart disease or stroke; FRS, Framingham risk score; major coronary event, myocardial infarction or death due to coronary heart disease.
The effect of the FRS + ABI model on prediction, in comparison to the FRS, is shown in Figure 1 according to low, intermediate, and high 10-year risk for a major coronary event. Within each FRS risk category, the FRS + ABI model refined the degree of risk corresponding to trends in event rates (except for a low event rate in women classified at intermediate risk of 10–19% by both models).
Kaplan–Meier 10-year rates in men and women for major coronary events in risk categories predicted by the Framingham risk score with ankle brachial index and the Framingham risk score.
Reclassification in predicting major coronary events and cardiovascular mortality for the Framingham risk score with ankle brachial index compared to Framingham risk score alone in men and women
Values are % (95% CI). Results derived from external validation dataset. Data for primary outcome of major coronary events shown in Supplementary Tables 3 and 4. Reclassification up and down shows the percentage of persons in whom the 10-year risk category increased or decreased, respectively, between <10%, 10–19%, and ≥20% for major coronary events and <2%, 2–4%, and ≥5% for cardiovascular mortality
Based on crude event rates and not survival method because lack of subjects in low risk group did not permit accurate estimation
Cardiovascular mortality, death due to coronary heart disease or stroke; major coronary event, myocardial infarction or death due to coronary heart disease; NRI, net reclassification improvement: NRI = {(P(event|up) – P(event)) * P(up) + (P(event) – P(event|down)) * P(down)}/{P(event) * (1 – P(event)}, with P(event), P(event|up), and P(event|down) all estimated using the Kaplan–Meier approach.
Predicting events in subjects at intermediate risk
Reclassification in predicting major coronary events and cardiovascular mortality for the Framingham risk score with ankle brachial index compared to Framingham risk score alone in men and women at intermediate risk
Values are % (95% CI). Results derived from external validation dataset. Data for primary outcome of major coronary events shown in Supplementary Tables 3 and 4. Study subjects are only those in FRS 10-year intermediate risk categories of 10–19% for major coronary events and 2–4% for cardiovascular mortality. Reclassification up and down shows the percentage of persons in whom 10-year risk either increased above or decreased below intermediate risk category, respectively
Based on crude event rates and not survival method because probability estimates not feasible when no person reclassified down
Cardiovascular mortality, death due to coronary heart disease or stroke; major coronary event, myocardial infarction or death due to coronary heart disease; NRI, net reclassification improvement: NRI = {(P(event|up) – P(event)) * P(up) + (P(event) – P(event|down)) * P(down)}/{P(event) * (1 – P(event)}, with P(event), P(event|up), and P(event|down) all estimated using the Kaplan–Meier approach.
The impact of reclassification on major coronary events using the FRS + ABI model was analysed using a wider FRS intermediate 10-year risk category of 5–19% (Supplementary Tables 5 and 6). This categorization resulted in very few numbers in the <5% risk group. In the whole population, the NRI for men was modest (3.1% (95% CI 0.6 to 6.4%, p = 0.018)) but for women was considerable (20.4% (95% CI 11.6 to 22.5%, p < 0.001)), with improved net reclassification for those having and not having an event. Restricting the FRS + ABI model to the 5–19% intermediate group led to a higher NRI in men (7.9% (95% CI 3.7 to 11.5%, p < 0.001)) but a lower NRI in women (13.0% (95% CI 7.3 to 17.9%, p < 0.001)).
Predicting events using cardiovascular risk covariate model
C-indices for the newly developed risk factor model in predicting major coronary events in the external validation dataset were 0.683 (95% CI 0.611 to 0.748) in men and 0.788 (95% CI 0.709 to 0.850) in women, which were slightly higher in men and considerably higher in women than the corresponding FRS C-indices in Table 2. Incorporation of the ABI resulted in only a slight improvement, increasing C-indices to 0.690 (95% CI 0.618 to 0.754) in men and 0.791(95% CI 0.712 to 0.852) in women, with nonsignificant NRIs of 2.0% (95% CI –2.3 to 4.2%, p = 0.567) and 1.1% (95% CI –1.9 to 4.0%, p = 0.483), respectively. In only those at intermediate 10–19% risk, NRIs were 7.7% (95% CI 0.0 to 13.0%, p = 0.049) in men and 2.4% (95% CI –3.0 to 10.5%, p = 0.275) in women.
Discussion
Main findings
In this analysis combining data from 18 population-based studies, a new ABI risk model incorporating the FRS + ABI was developed and then validated in an external population. In comparison to the FRS, as measured by C-index and NRI, the ABI model led to an improvement in performance, but mostly in women. Restricting use of the ABI model to those at intermediate 10-year risk of 10–19% for major coronary events resulted in higher NRIs than in the whole population. In a wider intermediate risk group of 5–19%, the NRIs were lower than in the 10–19% risk group.
The more impressive results in women than in men might be related to the particularly poor performance of the FRS in women in the external validation dataset in which C-indices were much lower than in the development and internal validation datasets (Table 2). This may have been due to chance or unexpected population differences. The main effect of incorporating the ABI may have been to compensate for this poor performance by bringing the C-indices up towards those of men and not that the ABI model was inherently superior in women than in men. When a better performing model based on fitting individual risk covariates was used instead of the FRS, incorporation of the ABI in the prediction of major coronary events led to no significant improvement. These results suggest that the impact of the ABI is not a fixed phenomenon but is influenced by how well the base risk factor model performs.
Other studies
The Multi-Ethnic Study of Atherosclerosis (MESA) found that the ABI contributed to an improvement in classification of cardiovascular events and non-events compared to traditional risk factors as measured by integrated discrimination improvement. 34 In the Atherosclerosis Risk in Communities (ARIC) Study and Rotterdam Study, each components of the ABI Collaboration, the effect on prediction of adding the ABI to a cardiovascular risk equation was examined32,35 and found, in the whole study populations, to have minimal impact. In keeping with the present study, both the ARIC and Rotterdam studies found improved performance if the ABI was used only in patients at intermediate risk. In all men and women combined, the NRI was 0.8% in the ARIC Study and 0.6% in the Rotterdam Study and, in those at intermediate risk, was 8.3% and 7.3%, respectively. A similar result (NRI 7.0%) was found in the elderly aged 70–79 years at intermediate risk in the Health, Aging and Body Composition Study. 36 However, it has been pointed out recently that findings in intermediate groups may be overly optimistic due to possible bias in their estimation. 37
Comparison with coronary artery calcium
In the Rotterdam Study, coronary artery calcium (CAC) showed better prediction than the ABI when added to a locally developed risk factor model. 35 The NRIs for models in the whole population were 19.3% for CAC and 0.6% for ABI and, in those at intermediate risk, were 39.3% for CAC and 7.3% for ABI. Likewise in intermediate risk individuals in MESA, the NRI was much higher for CAC than for other measures of subclinical atherosclerosis including ABI. 36 However, measurement of CAC is technically sophisticated, expensive, and involves radiation exposure, 38 and so is less suited than the ABI to measurement in primary care or in population screening. The ABI can be measured easily with minimal training, using simple inexpensive equipment and in less than 15 min. 39 A risk prediction programme incorporating measures of subclinical disease might benefit from tiered assessment with more complex and costly tests targeted on fewer individuals. Research is required on the costs and effectiveness of such an approach.
Limitations
A principal strength of this study was that, due to large numbers of subjects, models could be tested in both internal and external validation datasets derived from 18 population studies from several countries. A limitation of using multiple studies is that measurement of variables, including ABI, and the ascertainment and definition of endpoints were not identical. However, studies were only included where consistent and valid methods were used. Furthermore, using two study outcomes with hard endpoints (major coronary events and cardiovascular mortality) and finding broadly similar results, provided some reassurance on validity. Nevertheless, very large simulated datasets have been used as an alternative approach to overcoming some of these difficulties. 40 A further limitation of the study was that the exclusion criteria at baseline of only subjects with coronary heart disease meant that some subjects with other cardiovascular disease would be included, but were likely to make up a small proportion of the total population. Insufficient data were available to study non-White populations.
The FRS applied in this study 25 has been adopted in many practice settings but, more recently, the Framingham group produced a general cardiovascular disease risk factor model to be used in primary care for predicting a wider range of events. 41 Due to limitations in our dataset, assessment of these outcomes was not feasible but, if predicting a wide range of events was desirable, evaluation of ABI risk models in this context would be useful, as would the contribution of the ABI to other risk models such as SCORE. 42
Clinical implications
The ABI has been recognized for some time as potentially useful in screening healthy individuals for cardiovascular risk. 43 However, guidelines have varied recommendations on use of the ABI. Recently, the US Preventive Services Task Force, while recognizing that use of the ABI would reclassify some men and women in the FRS intermediate risk category, considered the evidence insufficient to assess the benefits and harms, and concluded that clinical discretion was required. 44 On the other hand, an American College of Cardiology Foundation/American Heart Association guideline included a recommendation, based on level of evidence B (nonrandomized studies), that ‘Measurement of ABI is reasonable for cardiovascular risk assessment in asymptomatic adults at intermediate risk’. 45 Recent multi-Society European guidelines had a similar recommendation. 46 The results of our study provide some support for the use of an ABI risk model especially in individuals at intermediate risk and when performance of the base risk factor model is modest. Furthermore, if physicians are uncertain about how well the FRS performs in their practice, using the ABI model is likely to compensate for any deficiencies in the FRS.
Footnotes
Acknowledgements
The Atherosclerosis Risk in Communities study is carried out as a collaborative study. The authors thank the staff and participants of the ARIC study for their important contributions.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
