Risk prediction of incident coronary heart disease in the Netherlands: re-estimation and improvement of the SCORE risk function

Abstract

Aims: To re-estimate the SCORE risk function using individual data on risk factors and coronary heart disease (CHD) incidence from the Dutch Cardiovascular Registry Maastricht (CAREMA) population-based cohort study; to evaluate changes that may improve risk prediction after re-estimation; and to compare the performance of the resulting CAREMA risk function with that of existing risk scores.

Methods and results: The cohort consisted of 21,148 participants, born in 1927–1977 and randomly sampled from the Maastricht region in 1987–1997. After follow-up (median 10.9 years), 783 incident CHD cases occurred. Model performance was assessed by discrimination and calibration. The additional value of including other risk factors or current risk factors in a different manner was evaluated using the net reclassification index (NRI). The c statistic of the re-estimated SCORE model was 0.799 (95% CI 0.782–0.816). Separating the total/high-density lipoprotein (HDL) cholesterol ratio into total and HDL cholesterol levels did not improve the c statistic (p = 0.22), but reclassified 6.0% of the participants into a more appropriate risk category (p < 0.001) compared with the re-estimated model. The resulting CAREMA function reclassified 28% of the participants into a more appropriate risk category than the Framingham score. Compared with the SCORE functions for high- and low-risk regions, the NRIs were 28% and 35%, respectively, which can largely be explained by the difference in outcome definition (CHD incidence vs. CHD mortality).

Conclusion: In this Dutch population, a re-estimated SCORE function with total and HDL cholesterol levels instead of the cholesterol ratio can be used for the risk prediction of CHD incidence.

Keywords

Coronary heart disease prospective cohort study risk factors risk prediction

Introduction

Coronary heart disease (CHD) is a multifactorial disease associated with several risk factors that may also interact to increase risk. Therefore, the risk of CHD based on the combined effect of risk factors is more meaningful than the risk based on a single risk factor. Consequently, risk functions have been developed for cardiovascular disease (CVD)¹^–⁵ or CHD⁶^–¹² that estimate the risk in time taking into account the cumulative effect of risk factors. Such risk scores can be useful in the primary prevention of CVD/CHD by identifying persons at high risk of developing the disease.

In most European countries including the Netherlands, the SCORE function is often used in the primary prevention of CVD.^13,14 Separate risk functions have been developed for high- and low-risk regions. These functions estimate the 10-year risk of fatal CVD using the following predictors: age, sex, current smoking, total/high-density lipoprotein (HDL) cholesterol ratio, and systolic blood pressure.⁴ Some studies have found that applying these formulas to their own population overestimated absolute risks.¹⁵^–¹⁸ Therefore, several countries including the Netherlands have recalibrated the SCORE function using national estimates of incidence rates and mean risk factor levels. However, the problem with such aggregated data is that the risk factor and disease status are not measured at an individual level. Thus, it would be better to use observed individual data from prospective cohort studies for recalibration. Additionally, the SCORE functions estimate the risk of CVD mortality only, while for clinical practice the prediction of both fatal and non-fatal events is more relevant.

Furthermore, some risk factors such as physical inactivity and body mass index are mostly not included in current risk scores probably due to their limited availability or their limited added value to risk prediction as assessed by the discriminative ability of risk scores. Many studies have investigated whether the inclusion of additional risk factors resulted in an improvement of the risk prediction.¹⁹^–²⁷ They found that adding predictors resulted in only small improvements in the predictive ability of risk scores using the discriminative ability (determined as the area under the receiver operating characteristic curve) as the criterion. However, adding these factors may still improve risk prediction when evaluated by reclassification of subjects to a more appropriate risk category.²⁸

Therefore, the goals of this study were (1) to re-estimate the SCORE risk function using individual data on risk factors and both fatal and non-fatal CHD incidence from the Cardiovascular Registry Maastricht (CAREMA) cohort study in the Netherlands; (2) to evaluate whether including additional risk factors or current risk factors in a different manner into this re-estimated function improves risk prediction; and (3) to compare the risk prediction of CHD incidence by the re-estimated and extended risk function from the CAREMA cohort study with the risk predicted by functions of the SCORE project⁴ for high-risk (SCORE-high) and low-risk regions (SCORE-low) and the Framingham risk score.¹⁰

Methods

Study population

The study participants, living in the Maastricht region, were derived from two large monitoring projects in the Netherlands: the Monitoring Project on Cardiovascular Disease Risk Factors (PPHVZ) 1987–1992²⁹ and the Monitoring Project on Chronic Disease Risk Factors (MORGEN) 1993–1997.³⁰ Each year, a random sample of people aged 20–59 years was selected from municipal registries of Maastricht and surrounding communities, i.e. Eijsden, Margraten, Meerssen, and Valkenburg aan de Geul. Between 1987 and 1997, 21,662 participants, born between 1927 and 1977, were included in this study.

Baseline data

At baseline, all participants completed a self-administered questionnaire on demographic characteristics, medical history, parental history of myocardial infarction, and life style factors such as smoking, alcohol consumption, usual dietary intake, and physical activity. In addition, all participants underwent a physical examination at the Municipal Health Service to measure their height, weight, and blood pressure.^29,30 All measurements were carried out by trained staff members who were instructed by the same physician. Non-fasting blood samples were taken for the determination of total and HDL cholesterol levels.

Follow-up

Of the 21,662 participants, 21,148 (97.6%) had given written informed consent to retrieve information from the Municipal Population Registries and their general practitioner or specialist. A migration and mortality follow-up was performed by record linkage of the cohort to the Municipal Population Registries. During follow-up until 31 December 2003, 791 participants (3.7%) died, 621 (2.9%) emigrated, 2,106 (10.0%) migrated to a municipality outside the Maastricht region, and 12 (0.1%) were lost to follow-up.

Cardiological follow-up was performed by record linkage to the Cardiology Information System (CIS) of the University Hospital Maastricht and to the causes of death registry of Statistics Netherlands. Incident CHD was defined as a clinical diagnosis of an acute myocardial infarction, unstable angina pectoris, a percutaneous transluminal coronary angioplasty, or coronary artery bypass grafting according to the CIS or CHD as primary or secondary cause of death according to Statistics Netherlands (ICD9 410–414 or ICD10 I20–I25).

Person time at risk was calculated from baseline until end of follow-up, i.e. a clinical diagnosis of CHD, death, emigration, migration outside the Maastricht region, or censoring at 31 December 2003, whichever occurred first. Participants with CHD at baseline (n = 347), those who migrated to a municipality outside the Maastricht region before baseline (n = 26) or who were lost to follow-up (n = 12) were excluded from the analyses.

Statistical analyses

All analyses were performed using the Stata statistical software package 9.2 (Stata Corporation, College Station, TX, USA). Two-sided p-values are reported throughout this paper, with p < 0.05 considered statistically significant.

Re-estimation

Weibull proportional hazards models with age as time variable were used for the development of the prediction models, similar as in the SCORE project.⁴ Separate baseline survival functions were obtained for men and women while the coefficients for the predictors were obtained from the total cohort. Continuous predictors were centred and linearity was checked using restricted cubic spline analyses.

The proportional hazard assumption was tested by checking the log–log curves. To see whether the assumptions about the underlying survival functions made by the Weibull model influenced the risk prediction, the Weibull model was cross-checked with a Cox regression model. Based on previous risk functions, interaction terms were tested between total/HDL cholesterol ratio and sex,³¹ and between systolic blood pressure and use of blood pressure-lowering medication.³²

For the re-estimation the same predictors were included as in the SCORE risk function, i.e. age in years (time variable), sex (strata), current smoking (yes/no), systolic blood pressure (mmHg), and total/HDL cholesterol ratio. Because of the low number of missing values and negligible impact on the regression coefficients, participants with missing data on at least one predictor (n = 619) were excluded from the analysis. In addition, participants with inconsistent data on smoking or alcohol consumption (n = 89) were excluded, leaving 20,055 participants including 783 incident CHD cases.

Model performance

Model performance was evaluated in the study population of 40 years and older at baseline. Discrimination was evaluated using the c statistic which is the probability of the model to distinguish between those who developed CHD during the follow-up period and those who did not, taking into account censored follow-up times. Calibration was used to assess the agreement between observed and predicted risks. The 10-year predicted risk of incident CHD was calculated as described in the Appendix (available online). In each decile of predicted risk, the observed 10-year risk was calculated from the 10-year Kaplan–Meier estimate and the ratio between predicted and observed risk was calculated. A calibration plot was made by plotting the observed risk against the predicted risk. The calibration slope and intercept were calculated using a linear regression model with the observed risk as the outcome variable and the predicted risk as the only independent variable. In case of perfect agreement, these estimates are one and zero, respectively. Because the interpretation of the intercept is difficult when the slope is unequal to one, the intercept was calculated with the slope fixed at one. Model performance was assessed in participants of 40 years and older at baseline that were part of the total study population in which the model was developed. Formal internal validation of the model by bootstrapping was not considered indicated, because of the very favourable ratio of events per variable (783/8).^33,34

Evaluating improvements in risk prediction

Several changes to the re-estimated SCORE function were investigated to assess their contribution to risk prediction. Besides discrimination and calibration, the prediction models were compared using the net reclassification index (NRI) taking into account persons with incomplete 10-year follow-up.^28,35 This estimate refers to the percentage of subjects shifting to a more appropriate risk category (to a higher risk category for cases, to a lower risk category for non-cases) when using the new risk function compared with the original function. According to the predicted risks, subjects were divided in the following risk categories: 0–2%, 2–5%, 5–10%, and ≥10%.³⁶ The final model from these analyses is referred to as the CAREMA risk function.

The performance of the CAREMA risk function to predict CHD incidence was compared to the performance of SCORE-high and SCORE-low,⁴ as these risk functions are mostly used for primary prevention of CVD in Europe, including the Netherlands. However, since the CAREMA risk function predicts the risk of incident CHD, we only used part of the SCORE formulas that predicts the 10-year risk of CHD mortality. Nevertheless, because these SCORE functions predict CHD mortality instead of CHD incidence, the CAREMA risk function was also compared to the Framingham risk function which predicts the same endpoint.¹⁰ For these comparisons, the original formulas and regression coefficients of these risk functions were applied to the population of the CAREMA cohort study.

Results

During follow-up of maximally 16.9 years, 783 participants developed incident CHD (573 men, 210 women). Mean age at diagnosis was slightly lower for men than for women (57.0 and 58.7 years, respectively). Table 1 shows the baseline characteristics of the CAREMA study population.

Table 1.

Baseline characteristics of the Dutch Cardiovascular Registry Maastricht (CAREMA) cohort study, 1987–1997

	Total cohort (n = 20,055)
Age at baseline (years)	41.0 ± 10.9
Sex
Men	46.8 (9380)
Women	53.2 (10,675)
Baseline cohort
PPHVZ	69.4 (13,910)
MORGEN	30.6 (6145)
Current smoking	39.8 (7985)
No. of cigarettes/day^a	15.7 ± 8.9
Total cholesterol (mmol/l)	5.4 ± 1.1
HDL cholesterol (mmol/l)	1.3 ± 0.3
Total/HDL cholesterol ratio	4.3 (3.4–5.4)
Systolic blood pressure (mmHg)	119.2 ± 14.4
Diastolic blood pressure (mmHg)	76.4 ± 9.6
Pulse pressure (mmHg)	42.8 ± 9.6
Parental history of MI	30.0 (6020)
Father ever MI	23.1 (4637)
Age at diagnosis father (years)	60.4 ± 10.8
Mother ever MI	9.9 (1984)
Age at diagnosis mother (years)	65.0 ± 11.1
Body mass index (kg/m²)	24.2 (22.0–26.8)
Diabetes (%)	0.9 (175)
Physical activity during leisure time^b
None to light activity	43.8 (8778)
Moderate to heavy activity	56.2 (11,269)
Regular alcohol intake (≥1 glass/week)	62.5 (12,497)
No. of glasses consumed per day^c	1.4 (0.7–2.4)

Data represent mean ± SD, % (n), or median (25–75th percentile). HDL, high-density lipoprotein; MI, myocardial infarction; MORGEN, Monitoring Project on Chronic Disease Risk Factors; PPHVZ, Monitoring Project on Cardiovascular Disease Risk Factors. ^aRestricted to current smokers. ^bNumbers may not add up to the total number of cohort members because of missing values. ^cRestricted to regular drinkers (≥1 glass/week).

Our re-estimated SCORE risk function had a c statistic of 0.799 (95% CI 0.782–0.816) (Table 2). The calibration slope and intercept were 1.06 and 0.09%, respectively. The model performance significantly decreased after including age as predictor instead of as time variable or using linear transformations for the continuous predictors (data not shown). Although the c statistic increased to 0.806 when parental history of myocardial infarction was added to the model, none of the changes significantly increased the c statistic (Table 2). Based on the NRI, however, replacement of the total/HDL cholesterol ratio by total and HDL cholesterol separately (Table 2, model 3A and 3B) improved model performance of the re-estimated SCORE function (p < 0.001). Furthermore, this change resulted in slightly better calibration compared with the re-estimated SCORE function (model 1). Including an interaction term between HDL cholesterol and sex (model 3B) did not improve the predictive value significantly compared with model 3A. Therefore, the simplest model is preferred. Thus, the CAREMA risk function consists of the following predictors: age in years (time variable), sex (strata), current smoking (yes/no), systolic blood pressure (continuous), total cholesterol (continuous), and HDL cholesterol (continuous). This risk function had a good discriminative ability (c statistic 0.802, 95% CI 0.785–0.819) and calibration (slope 1.00, intercept 0.07%). In Figure 1, the calibration plot and the ratio for predicted/observed risk for the CAREMA risk function are shown.

Figure 1.

Calibration plot of the CAREMA risk function (slope 1.00; intercept 0.07%), including an overview of the predicted/observed risk ratio (restricted to the study population of ≥40 years at baseline).

Table 2.

Changes in the performance of the re-estimated SCORE prediction model using individual data from the Dutch Cardiovascular Registry Maastricht (CAREMA) cohort study according to several adaptations

		Discriminative ability			Calibration		Reclassification
Model specification	N	c statistic	95% CI	p ^a	Slope	Intercept (%)	NRI (%)	p ^a
Model 1: Re-estimated SCORE model including age (time variable), sex (strata), total/HDL cholesterol ratio (cont), systolic blood pressure (cont), and current smoking (yes/no)	11,250	0.799	0.782–0.816	–	1.06	0.09	–	–
Current predictors included in a different manner
Model 2: Model 1 but instead of current smoking
A: 3 categories: never, ex, current	11,250	0.800	0.783–0.817	0.54	1.07	0.08	0.68	0.45
B: 3 categories including time since quitting^b	11,162	0.800	0.783–0.817	0.52	1.07	0.09	0.97	0.33
C: No. of cigarettes/day: none, <1–19, ≥20^c	11,213	0.801	0.784–0.818	0.17	1.07	0.10	–0.84	0.36
Model 3: Model 1 but instead of cholesterol ratio
A: Total and HDL cholesterol separately	11,250	0.802	0.785–0.819	0.22	1.00	0.07	5.99	<0.001
B: Model 3A + interaction sex × HDL cholesterol	11,250	0.802	0.785–0.819	0.22	0.99	0.07	5.66	<0.001
Model 4: Model 1 but instead of systolic blood pressure
A: Pulse pressure	11,250	0.793	0.775–0.810	0.004	1.09	0.17	−0.90	0.57
Addition of potential predictors
Model 5: Model 1 + parental history of MI
A: Parental history (yes/no)	10,979	0.806	0.790–0.823	0.13	1.06	0.04	2.78	0.15
B: No. of parents with MI: 0, 1, or 2	10,891	0.807	0.791–0.824	0.09	1.05	0.01	2.42	0.21
C: Parental history of premature MI (yes/no)^d	10,827	0.806	0.789–0.822	0.30	1.07	0.06	1.91	0.20
D: No. of parents with premature MI: 0, 1, or 2^d	10,804	0.806	0.789–0.822	0.23	1.07	0.06	1.60	0.29
E: Parental history of premature MI: no parents affected, non-premature MI, premature MI^d,e	10,806	0.807	0.791–0.824	0.11	1.05	0.04	2.57	0.18
Model 6: Model 1 + body mass index
A: Continuous	11,245	0.799	0.782–0.816	1.00	1.05	0.05	1.53	0.15
B: WHO categories^f	11,245	0.799	0.782–0.816	0.69	1.06	0.06	0.17	0.85
Model 7: Model 1 + diabetes (yes/no)	11,229	0.801	0.784–0.818	0.20	1.05	0.08	1.55	0.07
Model 8: Model 1 + physical activity (leisure time)^g	11,246	0.799	0.782–0.816	0.45	1.05	0.08	−0.61	0.10
Model 9: Model 1 + total alcohol consumption^h	11,217	0.801	0.784–0.818	0.21	1.04	0.07	0.70	0.55

CI, confidence interval; cont, continuous; HDL, high-density lipoprotein; MI, myocardial infarction; NRI, Net Reclassification Index; WHO, World Health Organization. Assessment of model performance was restricted to the study population of ≥40 years at baseline. ^aComparing the adapted models with model 1 using the same number of observations. ^bCategories: never smoker or ex-smoker quitted longer than 5 years; ex-smoker quitted within 5 years; and current smoker. ^cThe dichotomous variable for current smoking was excluded from model 2C because of collinearity with the categorical variable for the number of cigarettes/day. ^dAge at diagnosis ≤60 years. ^eAt least one parent with a premature MI. ^fUnderweight (<18.5 kg/m²), normal weight (18.5–24.9 kg/m²), overweight (25.0–29.9 kg/m²), and obesity (≥30.0 kg/m²). ^gNone to light vs. moderate to heavy physical activity in leisure time. ^hMen: never or ex-drinker, 0–3 glasses/day, >3 glasses/day; Women: never or ex-drinker; 0–2 glasses/day; >2 glasses/day.

Using the CAREMA risk function instead of SCORE-high or SCORE-low, the NRIs were 28% and 35%, respectively (p < 0.001) (Table 3). Both SCORE-high and SCORE-low underestimated the 10-year absolute risks of incident CHD in the CAREMA study population, as expected given the fact that the SCORE formulas that were used predicted the 10-year risk of fatal CHD, and not non-fatal CHD. More details about the differences between the CAREMA and SCORE risk functions are given in the Appendix. The Framingham risk score, however, which predicts the 10-year risk of both fatal and non-fatal CHD, overestimated the absolute risks in this study population. Using the CAREMA risk function instead of the Framingham score resulted in an NRI of 28% (p < 0.001).

Table 3.

Comparison of the performance to predict coronary heart disease incidence between the Dutch Cardiovascular Registry Maastricht (CAREMA) risk function and existing risk scores in the CAREMA study population

Risk score	Discriminative ability			Calibration		Reclassification
Risk score	c statistic	95% CI	p ^a	Slope	Intercept (%)	NRI (%)	p ^a
CAREMA	0.802	0.785–0.819	–	1.00	0.07	–	–
SCORE-high⁴	0.789	0.771–0.807	<0.001	2.32	3.29	−27.91	<0.001
SCORE-low⁴	0.789	0.771–0.807	<0.001	4.63	4.19	−34.63	<0.001
Framingham study¹⁰	0.792	0.774–0.810	0.01	0.82	−4.14	−28.05	<0.001

CI, confidence interval; NRI, Net Reclassification Index. Assessment of model performance was restricted to the study population of ≥40 years at baseline. ^aCompared to the CAREMA risk function using the same number of observations.

Discussion

After re-estimation of the SCORE risk function using individual data on risk factor levels and CHD incidence from the CAREMA cohort study, risk prediction of CHD incidence can be improved by separating the total/HDL cholesterol ratio into total and HDL cholesterol levels. The resulting CAREMA risk function classified 28% of the participants into a more appropriate risk category for CHD incidence compared with the Framingham risk score. Compared with SCORE-high and SCORE-low, the NRIs were 28% and 35%, respectively, which can largely be explained by the difference in disease outcome.

The SCORE risk function calculates the 10-year risk of CVD mortality only.⁴ In practice, however, there is at least equal interest in the prevention of non-fatal CVD. Therefore, the endpoint of the SCORE function was extended to include also incident non-fatal CHD cases in this study. In the CAREMA cohort study, hospital-based data on non-fatal CHD events were collected in an extensive manner so the cardiological follow-up is expected to be nearly complete and misclassification of the disease is expected to be low.³⁷ On the other hand, the endpoint definition had to be restricted to the incidence of CHD instead of CVD, as accurate data on other CVD events such as stroke were not available from the cardiology register. Nevertheless, this difference in endpoint has contributed to the high percentages of both cases and non-cases shifting to a higher risk category when the CAREMA function was used instead of SCORE-high or SCORE-low, while none of the participants were reclassified into a lower risk category (Appendix). For CHD incidence, however, these higher predicted risks are more appropriate. In addition, higher cut-off points for risk stratification are used in the prevention of CVD incidence than of CVD mortality.¹⁴

In the present study, only a small proportion of the study population reported the use of medication for high blood pressure (4.5%) and/or high cholesterol level (0.5%) at baseline. Among previous risk scores studies, only three^1,2,11 evaluated blood pressure-lowering medication as potential predictor, of which two studies actually included it in their risk function.^1,11 In the present study, systolic blood pressure predicted the CHD risk similarly in participants with and without blood pressure-lowering medication (p interaction = 0.07). No information was available about the start of medication use during the follow-up period. Because medication use may prevent or postpone the occurrence of CHD, inclusion of participants who started to use medication between baseline and end of follow-up may have led to an underestimation of the predicted risk.³⁸

Many studies found that adding risk factors to current risk scores resulted in no or only small improvements in risk prediction as assessed by the area under the receiver operating characteristic curve (equivalent to the c statistic).¹⁹^–²⁷ This estimate, however, may be insensitive to detect clinically relevant improvements in risk prediction when the traditional risk factors are already included.^28,39 Therefore, in addition to the c statistic, the NRI can be estimated which is probably better qualified for this purpose.²⁸ Most changes evaluated in this study did not significantly improve risk prediction, regardless whether the c statistic or NRI was used. However, based on the NRI, separating the cholesterol ratio into total and HDL cholesterol levels resulted in a small but significant improvement in risk prediction by the re-estimated SCORE function (NRI 6.0%). This was also reported in a recent publication by the SCORE investigators,³⁶ although they found a lower NRI (3.8%).

In primary practice in the Netherlands, the Framingham risk score has been used for CVD prevention in the past, while it is nowadays replaced by the SCORE risk charts. In the present study population, the Framingham risk score overestimated the 10-year absolute risk of CHD, as was also found in other studies.⁴⁰ This can be explained by the higher CHD incidence and mortality and the higher prevalence of risk factors at the time the Framingham risk score was developed. Because of this, the SCORE project developed risk functions for high- and low-risk regions based on European data, but predicted the 10-risk of fatal CVD instead of total CVD. Although SCORE-low can be used quite well to predict the risk of CVD mortality in the Netherlands,⁴¹ the results of our study show that both SCORE-high and SCORE-low are inappropriate to predict the 10-year risk of CHD incidence. As expected, the CAREMA risk function performed better as it is especially developed to predict this endpoint. Nevertheless, the CAREMA function also performed better than the Framingham risk score, which predicts the same endpoint. The CAREMA function, however, has a home advantage as its performance is evaluated in the population in which the function is developed. Although the CAREMA function may be a useful tool to predict the 10-year risk of CHD incidence in the Netherlands, its performance has first to be validated externally as well.

In conclusion, a re-estimated SCORE risk function with total and HDL cholesterol separately instead of the cholesterol ratio can be used for the risk prediction of CHD incidence in this Dutch population. Although the CAREMA risk function which includes these changes performed better than the SCORE and Framingham risk function in this study population, external validation of the function is necessary before use in practice.

Footnotes

Acknowledgements

The authors wish to thank D Kromhout for supervision of the PPHVZ project and the MORGEN project steering committee [HB Bueno de Mesquita, HA Smit, WMM Verschuren, JC Seidell (project leader)], epidemiologists and field workers of the Municipal Health Service in Maastricht for their contribution to baseline data collection and those involved in the logistics management (A Jansen, J Steenbrink) and the data management (A Blokstra, A van Kessel, P Steinberger, E den Hoedt, I Toxopeus, J van der Laan). The authors further wish to thank D Jaspers, A Hilton, V Visser, P Erkens, S Philippens, J Bremen, B Bleijlevens, T van Moergastel, and S van de Crommert for assistance in clinical data collection. The authors thank Dr AP Fitzgerald and Dr MT Cooney for providing more detailed information on the SCORE coefficients. Statistics Netherlands is acknowledged for providing data on causes of death.

Funding

This work was supported by CAPHRI School for Public Health and Primary Care. The Monitoring Project on Cardiovascular Disease Risk factors (PPHVZ) and the MORGEN study were financially supported by the Ministry of Public Health, Welfare and Sports of the Netherlands.

Conflict of interest

None declared.

References

Hippisley-Cox

Coupland

Vinogradova

Robson

Minhas

Sheikh

. Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ 2008; 336(7659): 1475–1482.

Ridker

Buring

Rifai

Cook

. Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA 2007; 297(6): 611–619.

Woodward

Brindle

Tunstall-Pedoe

. Adding social deprivation and family history to cardiovascular risk assessment: the ASSIGN score from the Scottish Heart Health Extended Cohort (SHHEC). Heart 2007; 93(2): 172–176.

Conroy

Pyorala

Fitzgerald

Sans

Menotti

De Backer

. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J 2003; 24(11): 987–1003.

Balkau

Qiao

Tuomilehto

Borch-Johnsen

Pyorala

. Prediction of the risk of cardiovascular mortality using a score that includes glucose as a risk factor. The DECODE Study. Diabetologia 2004; 47(12): 2118–2128.

Shaper

Pocock

Phillips

Walker

. Identifying men at high risk of heart attacks: strategy for use in general practice. Br Med J (Clin Res Ed) 1986; 293(6545): 474–479.

Tunstall-Pedoe

. The Dundee coronary risk-disk for management of change in risk factors. BMJ 1991; 303(6805): 744–747.

Menotti

Lanti

Puddu

Kromhout

. Coronary heart disease incidence in northern and southern European populations: a reanalysis of the seven countries study for a European coronary risk chart. Heart 2000; 84(3): 238–244.

Stevens

Kothari

Adler

Stratton

. The UKPDS risk engine: a model for the risk of coronary heart disease in Type II diabetes (UKPDS 56). Clin Sci (Lond) 2001; 101(6): 671–679.

10.

Wilson

D’Agostino

Levy

Belanger

Silbershatz

Kannel

. Prediction of coronary heart disease using risk factor categories. Circulation 1998; 97(18): 1837–1847.

11.

Chambless

Folsom

Sharrett

Sorlie

Couper

Szklo

. Coronary heart disease risk prediction in the Atherosclerosis Risk in Communities (ARIC) study. J Clin Epidemiol 2003; 56(9): 880–890.

12.

Assmann

Schulte

Cullen

Seedorf

. Assessing risk of myocardial infarction and stroke: new data from the Prospective Cardiovascular Munster (PROCAM) study. Eur J Clin Invest 2007; 37(12): 925–932.

13.

Graham

Atar

Borch-Johnsen

Boysen

Burell

Cifkova

. European guidelines on cardiovascular disease prevention in clinical practice: executive summary: Fourth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (Constituted by representatives of nine societies and by invited experts). Eur Heart J 2007; 28(19): 2375–2414.

14.

Smulders

Burgers

Scheltens

van Hout

Wiersma

Simoons

. Clinical practice guideline for cardiovascular risk management in the Netherlands. Neth J Med 2008; 66(4): 169–174.

15.

Ulmer

Kollerits

Kelleher

Diem

Concin

. Predictive accuracy of the SCORE risk function for cardiovascular disease in clinical practice: a prospective evaluation of 44 649 Austrian men and women. Eur J Cardiovasc Prev Rehabil 2005; 12(5): 433–441.

16.

Neuhauser

Ellert

Kurth

. A comparison of Framingham and SCORE-based cardiovascular risk estimates in participants of the German National Health Interview and Examination Survey 1998. Eur J Cardiovasc Prev Rehabil 2005; 12(5): 442–450.

17.

Bhopal

Fischbacher

Vartiainen

Unwin

White

Alberti

. Predicted and observed cardiovascular disease in South Asians: application of FINRISK, Framingham and SCORE models to Newcastle Heart Project data. J Public Health (Oxf) 2005; 27(1): 93–100.

18.

Lindman

Veierod

Pedersen

Tverdal

Njolstad

Selmer

. The ability of the SCORE high-risk model to predict 10-year cardiovascular disease mortality in Norway. Eur J Cardiovasc Prev Rehabil 2007; 14(4): 501–507.

19.

Atsma

van der Schouw

Grobbee

Hoes

Bartelink

. No added value of age at menopause and the lifetime cumulative number of menstrual cycles for cardiovascular risk prediction in postmenopausal women. Int J Cardiol 2008; 130(2): 190–195.

20.

Cooper

Miller

Bauer

Morrissey

Meade

Howarth

. Comparison of novel hemostatic factors and conventional risk factors for prediction of coronary heart disease. Circulation 2000; 102(23): 2816–2822.

21.

Lee

Price

Russell

Smith

van Wijk

Fowkes

. Improved prediction of fatal myocardial infarction using the ankle brachial index in addition to conventional risk factors: the Edinburgh Artery Study. Circulation 2004; 110(19): 3075–3080.

22.

Lloyd-Jones

Liu

Tian

Greenland

. Narrative review: Assessment of C-reactive protein in risk prediction for cardiovascular disease. Ann Intern Med 2006; 145(1): 35–42.

23.

Nawrot

Staessen

Thijs

Fagard

Tikhonoff

Wang

. Should pulse pressure become part of the Framingham risk score?. J Hum Hypertens 2004; 18(4): 279–286.

24.

van der Steeg

Boekholdt

Stein

El-Harchaoui

Stroes

Sandhu

. Role of the apolipoprotein B–apolipoprotein A-I ratio in cardiovascular risk assessment: a case-control analysis in EPIC-Norfolk. Ann Intern Med 2007; 146(9): 640–648.

25.

Wang

Gona

Larson

Tofler

Levy

Newton-Cheh

. Multiple biomarkers for the prediction of first major cardiovascular events and death. N Engl J Med 2006; 355(25): 2631–2639.

26.

Wilsgaard

Arnesen

. Body mass index and coronary heart disease risk score: the Tromso study, 1979 to 2001. Ann Epidemiol 2007; 17(2): 100–105.

27.

Ridker

Paynter

Rifai

Gaziano

Cook

. C-reactive protein and parental history improve global cardiovascular risk prediction: the Reynolds Risk Score for men. Circulation 2008; 118(22): 2243–2251, 4p following 2251.

28.

Pencina

D’Agostino

Sr D’Agostino

Jr Vasan

. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008; 27(2): 157–172, discussion 207–212.

29.

Van Leer

Seidell

Kromhout

. Dietary calcium, potassium, magnesium and blood pressure in the Netherlands. Int J Epidemiol 1995; 24(6): 1117–1123.

30.

Lean

Han

Seidell

. Impairment of health and quality of life in people with large waist circumference. Lancet 1998; 351(9106): 853–856.

31.

Boland

Christiaens

Goderis

Govaerts

Philips

Smeets

. Globaal cardiovasculair risicobeheer. Huisarts Nu 2007; 36(7): 339–369.

32.

Hippisley-Cox

Coupland

Vinogradova

Robson

May

Brindle

. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ 2007; 335(7611): 136–136.

33.

Steyerberg

Eijkemans

Harrell

Jr Habbema

. Prognostic modeling with logistic regression analysis: in search of a sensible strategy in small data sets. Med Decis Making 2001; 21(1): 45–56.

34.

Harrell

Jr Lee

Mark

. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996; 15(4): 361–387.

35.

Steyerberg

Pencina

. Reclassification calculations for persons with incomplete follow-up. Ann Intern Med 2010; 152(3): 195–196, author reply 196–197.

36.

Cooney

Dudina

De Bacquer

Fitzgerald

Conroy

Sans

. How much does HDL cholesterol add to risk estimation? A report from the SCORE Investigators. Eur J Cardiovasc Prev Rehabil 2009; 16(3): 304–314.

37.

Merry

Boer

Schouten

Feskens

Verschuren

Gorgels

. Validity of coronary heart diseases and heart failure based on hospital discharge and mortality data in the Netherlands using the cardiovascular registry Maastricht cohort study. Eur J Epidemiol 2009; 24(5): 237–247.

38.

Liew

Glasziou

. QRISK may be less useful. BMJ 2009; 339: b3485–b3485.

39.

Cook

. Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem 2008; 54(1): 17–23.

40.

Eichler

Puhan

Steurer

Bachmann

. Prediction of first coronary events with the Framingham score: a systematic review. Am Heart J 2007; 153(5): 722–731. 731.e1–8.

41.

van Dis

Kromhout

Geleijnse

Boer

Verschuren

. Evaluation of cardiovascular risk predicted by different SCORE equations: the Netherlands as an example. Eur J Cardiovasc Prev Rehabil 2010; 17(2): 244–249.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB