A multiple biomarker risk score for guiding clinical decisions using a decision curve approach

Abstract

Aims: We assessed whether a cardiovascular risk model based on classic risk factors (e.g. cholesterol, blood pressure) could refine disease prediction if it included novel biomarkers (C-reactive protein, N-terminal pro-B-type natriuretic peptide, troponin I) using a decision curve approach which can incorporate clinical consequences.

Methods and results: We evaluated whether a model including biomarkers and classic risk factors could improve prediction of 10 year risk of cardiovascular disease (CVD; chronic heart disease and ischaemic stroke) against a classic risk factor model using a decision curve approach in two prospective MORGAM cohorts. This included 7739 men and women with 457 CVD cases from the FINRISK97 cohort; and 2524 men with 259 CVD cases from PRIME Belfast. The biomarker model improved disease prediction in FINRISK across the high-risk group (20⊟40%) but not in the intermediate risk group, at the 23% risk threshold net benefit was 0.0033 (95% CI 0.0013−0.0052). However, in PRIME Belfast the net benefit of decisions guided by the decision curve was improved across intermediate risk thresholds (10⊟20%). At p_t= 10% in PRIME, the net benefit was 0.0059 (95% CI 0.0007⊟0.0112) with a net increase in 6 true positive cases per 1000 people screened and net decrease of 53 false positive cases per 1000 potentially leading to 5% fewer treatments in patients not destined for an event.

Conclusion: The biomarker model improves 10-year CVD prediction at intermediate and high-risk thresholds and in particular, could be clinically useful at advising middle-aged European males of their CVD risk.

Keywords

Cardiovascular prediction clinical utility decision curve discrimination statistic multiple biomarker score

Introduction

A large proportion of cardiovascular disease (CVD) risk can be explained by simple modifiable classic risk factors such as abnormal apolipoprotein levels, smoking, and hypertension but they cannot fully explain the distribution of risk in the general population. Primary prevention guidelines for CVD treatment rely on defining thresholds of risk in the population, with preventive therapies aimed at those identified to be at high risk (>20% 10-year risk of CVD) while those at low risk (0⊟5%) can avoid treatment and its potential side effects.^1,2 Although, a substantial part of the population attributable fraction of CVD risk occurs among those at intermediate risk (5⊟20%). However, prevention strategies aimed at all in the intermediate risk group may be too costly. More accurate predictions across a range of risk thresholds are needed and may help to reduce the burden of disease more efficiently. Several risk scores exist (e.g. Framingham risk score,³ PROCAM⁴) but the applicability and refinement of these risk score models to local populations can be complicated by variations in absolute risk, background characteristics of populations, and methodological and statistical challenges.^5,6 Decision making could be guided by additional information from emerging risk factors such as biomarkers, arterial function tests, or imaging tests of atherosclerosis.^7,8 Although many studies have shown promising associations between individual biomarkers and cardiovascular risk, their ability to enhance the performance of risk algorithms is modest and contested.^9,10 This has stimulated interest in combining multiple biomarkers together into novel risk scores to improve risk prediction.¹¹⁻¹³

More precise risk estimation is an important part of assessing the clinical utility of novel risk scores but the value of this information depends on how it is used to guide therapy and improve clinical outcomes. Recently there have been important contributions to both the statistical and clinical literature, expounding a more informative approach to the choice and use of prognostic models that could be used to tailor treatment in different patient groups.^14,15 Essentially, they remind us to consider at least four complementary aspects of model assessment, namely discrimination, calibration, external validation, and risk re-stratification.^1,15,16 None of these characteristics alone will provide a sufficient demonstration of clinical utility.¹⁷ Decision analysis can assess the clinical value of a model and improve standard statistical performance measures by incorporating costs and benefits.¹⁸ There are two general problems associated with applying traditional decision-analytic methods to prediction models. First, they require additional data, such as costs or quality-adjusted life years, often not included in the validation dataset. The second problem is that decision analysis requires a binary result from the evaluated test. Prediction models often provide a result in continuous form, such as the probability of an event from 0% to 100%. To evaluate such a model using decision-analytic methods, the continuous result must be dichotomized (or stratified) at a given threshold and then evaluated for a wide range of such thresholds. To overcome these two difficulties, the novel technique of decision curve analysis was developed. Decision curve analysis attempts to represent a range of threshold probabilities where a patient might opt for treatment depending on how they weigh up the relative harms of false positive and false negative results.¹⁸

The MORGAM project, a prospective population-based study, recently evaluated 30 biomarkers of CVD in 10,466 initially healthy individuals and developed a risk score based on three biomarkers for the prediction of CVD for primary prevention.¹⁹ This biomarker score combined high-sensitivity C-reactive protein (hsCRP), troponin I, and N-terminal pro-B-type natriuretic peptide (Nt-proBNP), with traditional risk factors, to improve 10-year absolute risk estimation for non-fatal and fatal CVD events in middle-aged European populations.¹⁹ The validation of the prediction model demonstrated significant improvement in discrimination (C index improvement from 0.67 to 0.7 with biomarkers, p= 0.004) and risk reclassification (with net reclassification improvement (NRI) 11%, p= 0.0008).¹⁹

In this paper, we wanted to assess this model’s performance using a decision curve approach, a method which complements the NRI in risk reclassification but can add to the assessment of the model’s utility by incorporating clinical consequences. This relatively new measure has not yet been widely used in practice¹⁵ and could aid in the assessment of our prognostic model for CVD.

Methods

Cohort description

The multinational MONICA, Risk, Genetics, Archiving, and Monograph project (MORGAM, www.ktl.fi/morgam) is a population-based prospective study of coronary events initially established to develop cardiovascular risk scores based on classic risk factors and to determine whether the addition of genetic or biomarker variation could improve the risk score. The project objectives and detailed descriptions of cohorts have been published elsewhere.^20,21 The endpoint is the first occurrence of a major cardiovascular event during the follow-up, which includes the first fatal or nonfatal definite or possible myocardial infarction or coronary death, unstable angina, cardiac revascularization, ischaemic stroke, or unclassifiable death. FINRISK and PRIME Belfast are two of 36 MORGAM cohorts which are standardized according to MORGAM criteria. The FINRISK97 cohort recruited men and women aged 25⊟74 years in 1997.²² The PRIME Belfast cohort recruited men aged 50⊟59 years during 1991⊟1994.²³ In both studies cardiovascular events were classified according to standard criteria,²¹ participants with a documented or self-reported history of a major cardiovascular event (e.g. hospitalized unstable angina, coronary bypass grafting, angioplasty) were excluded. Approval from the local research ethics committee was obtained and all subjects gave written informed consent. We examined 30 biomarkers and evaluated the discriminative ability of single biomarkers to improve risk prediction in the FINRISK and PRIME Belfast cohorts.¹⁹ A combination of three biomarkers performed better than any single biomarker at improving discrimination and calibration in the discovery set consisting of FINRISK men. These three biomarkers had a low level of correlation and represent different biological pathways; NT-proBNP is a marker of vascular function and neurohormonal activity, hsCRP is an inflammatory marker associated with plaque development and progression; and troponin I is a marker of myocardial injury. We tested the performance of this score in an independent sample PRIME Belfast men and FINRISK women which were not used for discovery of predictors and found it also improved prediction (external validation). Detailed analytical description of the hsCRP, Nt-proBNP, and troponin I measurements are reported elsewhere.¹⁹ A contemporary sensitive troponin I with a 10% coefficient of variation at the 99th population percentile was used.¹⁹

Statistical analysis

Missing biomarker data were handled using multiple imputation²⁴ using the WinBUGS software²⁵ with outcome information included in the imputation model to avoid attenuation of estimated effects in later analyses.²⁶ Due to their highly skewed distributions, troponin I, hsCRP, and Nt-ProBNP values were cubic root transformed in the analyses. We excluded participants on cholesterol medication at baseline from analysis.

Decision curve analysis is a novel method of evaluating prediction models which combines accuracy measures (sensitivity, specificity) and clinical applicability by considering the consequences associated with the test.¹⁸ Consider this in terms of a clinician’s binary decision to treat or not to treat a patient given an estimated 10-year risk of a disease, denoted p. Assuming no treatment, the patient may or may not develop the disease. A person who would develop the disease and is recommended treatment is deemed a true positive; the utility of this denoted as a. A false positive is a patient who is recommended treatment, but who would not develop the disease, even without treatment; the utility in this case is called b. Similarly, define false negative and true negative for those given a decision not to treat, and let their potential outcomes be c and d respectively. Now a−c is the benefit of a correct decision to treat, while b−d is the cost of unnecessary treatment. The choice of utilities has to weigh factors such as unnecessary drug treatment, with the potential for a rare but serious side effect of statin therapy ⊟ rhabdomyolysis ⊟ and the lifetime psychological impact of high-risk group inclusion. In the decision curve approach the risk threshold is determined by these utility measures. The expected outcome given treatment is pa+(1−p)b, and the expected outcome without treatment pc+(1−p)d. A threshold risk p_t guiding the treatment decision is obtained from the equation where the two expected outcomes with and without treatment are set equal. This yields p_t= 1/[(a−c)/(d−b)+1]. In other words, the optimal threshold p_t depends only on the ratio of the benefit of a correct decision to treat and the benefit of avoiding unnecessary treatment. Risks higher than p_t would mean that the treatment is expected to be beneficial. Following Vickers and Elkin,¹⁸ we fix the scale of utility such that a−c= 1 for a positive decision. Then the net benefit from using treatment threshold p_t for a population of n persons is (no. of true positives)/n × (a−c)⊟(no. of false positives)/n × (d−b)= (no. of true positives)/n⊟(no. of false positives)/n × (p_t/(1−p_t)), and hence depends on the utility measures only through the resulting threshold risk. In the decision curve approach, the prediction models are evaluated by presenting the net benefit values for a range of ratios of benefit of a correct decision to treat and the benefit of avoiding unnecessary treatment, or equivalently, for a range of risk thresholds.

We applied the decision curve approach to the FINRISK97 and PRIME Belfast data, comparing two prediction models. The classic risk factor model included blood pressure, high-density lipoprotein (HDL) cholesterol, non-HDL cholesterol (difference between total and HDL cholesterol), body mass index (BMI), daily smoking, and use of antihypertensive treatment at baseline, adjusted for age, sex, and area. We added the three biomarkers (hsCRP, troponin I, and Nt-proBNP) to the classic risk factor model to make the ‘biomarker model’. The risk estimates were calculated as 10-year cumulative incidences for the first CVD event, obtained from a competing risks survival model fitted for CVD and other (non-CVD) mortality.²⁷ The mortality was modelled using the same covariates as stated above. Proportions of true and false positives were calculated using the cumulative incidence function for CVD, following Vickers et al.²⁸ This approach also takes into account the censoring in time-to-event data.²⁸ Overfitting was avoided by calculating the risk estimates using 10-fold cross-validation.²⁹ The net benefit was presented as a decision curve, by plotting it over a range of threshold probabilities. Bootstrap resampling with 2500 replicates was used to obtain 95% confidence intervals for the difference in net benefit between the classic risk factor model and the biomarker model. All analyses were carried out using R³⁰ and its Survival library.

Results

Characteristics of the study cohorts

Following exclusion of individuals with prevalent CVD, there were 3753 men and 3986 women from FINRISK and 2524 men from PRIME Belfast used in the decision curve analysis. During follow-up, 320 men and 137 women in FINRISK and 259 men in PRIME Belfast experienced an incident fatal or non-fatal CVD event. In addition, there were 330 deaths due to causes other than CVD for men and women in FINRISK and 133 deaths for men in Belfast PRIME. The mean±SD age at baseline was 48.5 ± 13.6 years in men, 46.5 ± 12.9 years in women in FINRISK. In PRIME men it was 54.7 ± 2.9 years.

Cardiovascular prediction in FINRISK

The decision curve graph for FINRISK, with net benefit plotted across a range of threshold probabilities from 1% to 100%, is presented in Figure 1. The baseline classic risk factor model is nested with the biomarker model so any differences between the two curves indicate the added effect of the biomarkers. Both prediction models are useful (in comparison to treat all and treat none strategies) over the risk threshold range of ∼5−40%. For thresholds higher than this (>40%), the models no longer provide meaningful discrimination. For low thresholds (<5%), the benefit from using a model does not differ from the treat all strategy. The biomarker model has a higher net benefit than the classic risk factor model across a range of relatively high threshold probabilities from 20% to 40%. This corresponds to a ratio of 1.5 : 4, for example p_t= 20%, the benefit of a correct decision to treat is four times greater than the benefit of avoiding unnecessary treatment (Table 1). The lower graph in Figure 1 shows that collectively the net benefit difference is positive across this range with confidence intervals supporting a significant trend in the data. For example, using a risk threshold of 23%, the net benefit gain from using the biomarker model was 0.0033 (95% CI 0.0013⊟0.0052). This corresponds to a net decrease in the number of false positives of 11 per 1000 individuals and a net increase of 3 true positive cases of CVD per 1000 individuals (Table 1). In terms of clinical utility, use of the prediction model for a patient at p_t= 23% would lead to the equivalent of 1.1% fewer treatments in patients not destined for an event without any increase in the number of false negatives. For the intermediate range (p_t= 10⊟20%), the net benefit difference between the classic risk factor model and the biomarker model is marginal, with the curves largely overlapping.

Figure 1.

Decision curve, plotting net benefit against risk threshold probability of cardiovascular disease relative to treat all or treat none for FINRISK men and women. Old model refers to classic risk factor model; new model refers to the biomarker model (high-sensitivity C-reactive protein, troponin I, N-terminal pro-B-type natriuretic peptide, and classic risk factors). Lower graph shows net benefit variation over risk distribution with 95% confidence intervals.

Table 1.

Decision curve results for FINRISK97 men and women: comparison of the net benefit of cardiovascular disease prediction using the classic risk factor model and the biomarker model across a range of threshold probabilities (p_t) for FINRISK97 men and women

p_t (%)	(a−c)/(d−b)	Net benefit				Net increase in true positives (per 1000)	Net decrease in false positives (per 1000)
p_t (%)	(a−c)/(d−b)	Classic risk model	Biomarker model	Difference	95% confidence interval	Net increase in true positives (per 1000)	Net decrease in false positives (per 1000)
19	4.3	0.0100	0.0123	0.0023	0.0005 to 0.0041	2	10
20	4.0	0.0091	0.0107	0.0016	−0.0002 to 0.0036	2	7
21	3.8	0.0086	0.0096	0.0010	−0.0009 to 0.0028	1	4
22	3.5	0.0073	0.0089	0.0015	−0.0004 to 0.0034	2	5
23	3.3	0.0057	0.0089	0.0033	0.0013 to 0.0052	3	11
24	3.2	0.0051	0.0078	0.0027	0.0008 to 0.0046	3	8
25	3.0	0.0051	0.0068	0.0018	−0.0002 to 0.0037	2	5
26	2.8	0.0036	0.0063	0.0027	0.0008 to 0.0045	3	8
27	2.7	0.0039	0.0058	0.0019	0.0000 to 0.0037	2	5
28	2.6	0.0033	0.0050	0.0017	−0.0001 to 0.0035	2	4
29	2.4	0.0021	0.0045	0.0024	0.0004 to 0.0045	2	6
30	2.3	0.0020	0.0041	0.0021	0.0001 to 0.0041	2	5
31	2.2	0.0018	0.0044	0.0026	0.0006 to 0.0046	3	6
32	2.1	0.0018	0.0039	0.0021	0.0002 to 0.0040	2	4
33	2.0	0.0009	0.0039	0.0030	0.0011 to 0.0049	3	6
34	1.9	0.0007	0.0032	0.0025	0.0007 to 0.0045	3	5
35	1.9	0.0008	0.0032	0.0024	0.0007 to 0.0042	2	4
36	1.8	0.0010	0.0031	0.0021	0.0004 to 0.0039	2	4
37	1.7	0.0009	0.0026	0.0017	0.0000 to 0.0035	2	3
38	1.6	0.0008	0.0021	0.0013	−0.0004 to 0.0030	1	2
39	1.6	0.0004	0.0020	0.0016	0.0000 to 0.0033	2	2
40	1.5	0.0007	0.0012	0.0005	−0.0011 to 0.0021	0	1

Cardiovascular prediction in PRIME Belfast

The decision curve graph for PRIME Belfast is depicted in Figure 2. Both prediction models are useful (in comparison to treat all and treat none strategies) approximately over the risk threshold range of 5−40%. The net benefit of the biomarker model has positive values across the range p_t= 10⊟20, and at thresholds p_t= 10, 15, 18, 20, and 21, significant confidence intervals give some confidence to this trend. Across these points, the net benefit of true positives and negatives reclassify relatively large numbers in each threshold category. Although the results must be considered with caution because the confidence intervals are wide, calculating confidence intervals for 100 risk limits each using 2500 bootstrap replications on a dataset the size of PRIME can lead to a multiple testing problem and measurement error. In the context of the positive benefit of the biomarker model observed in the FINRISK cohort, the correlated result in the independent PRIME Belfast dataset is remarkable. Importantly, and in contrast to FINRISK, here the improvement is largest at threshold probabilities from 10% to 20%, encompassing the intermediate CVD risk range in PRIME Belfast. At p_t= 10%, the model with biomarkers performs well, giving a difference in the net benefit of 0.0059 (95% CI 0.0007⊟0.0112) (Table 2). This corresponds to a net increase of 6 true positive patients per 1000 and a net decrease of 53 false positive patients per 1000 are predicted (Table 2). Therefore, choice of this prediction model for patients at p_t= 10% would lead to the equivalent of 5% fewer treatments in patients not destined for an event without any increase in the number of false negatives, compared to the baseline model. Across the majority of thresholds, the improvement was in the number of patients who would have been falsely classified as positive with the classic risk factor model.

Figure 2.

Decision curve, plotting net benefit against risk threshold probability of cardiovascular disease relative to treat all or treat none for PRIME Belfast men. Old model refers to classic risk factor model; new model refers to the biomarker model (high-sensitivity C-reactive protein, troponin I, N-terminal pro-B-type natriuretic peptide, and classic risk factors). Lower graph shows net benefit variation over risk distribution with 95% confidence intervals.

Table 2.

Decision curve result for PRIME Belfast men: comparison of the net benefit of cardiovascular disease prediction using the classic risk factor model and the biomarker model across a range of threshold probabilities (p_t) for PRIME Belfast men

p_t (%)	(a−c)/(d−b)	Net benefit				Net increase in true positives (per 1000)	Net decrease in false positives (per 1000)
p_t (%)	(a−c)/(d−b)	Classic risk model	Biomarker model	Difference	95% confidence interval	Net increase in true positives (per 1000)	Net decrease in false positives (per 1000)
10	9.0	0.0233	0.0292	0.0059	0.0007 to 0.0112	6	53
11	8.1	0.0218	0.0267	0.0049	−0.0006 to 0.0102	5	40
12	7.3	0.0175	0.0221	0.0047	−0.0009 to 0.0100	5	34
13	6.7	0.0141	0.0179	0.0038	−0.0014 to 0.0091	4	25
14	6.1	0.0131	0.0147	0.0016	−0.0039 to 0.0070	2	10
15	5.7	0.0087	0.0143	0.0055	0.0005 to 0.0105	6	31
16	5.2	0.0097	0.0131	0.0034	−0.0015 to 0.0081	3	18
17	4.9	0.0097	0.011	0.0013	−0.0036 to 0.0061	1	6
18	4.6	0.0069	0.0115	0.0046	0.0002 to 0.0090	5	21
19	4.3	0.0052	0.0095	0.0043	−0.0003 to 0.0090	4	18
20	4.0	0.0033	0.0077	0.0044	0.0003 to 0.0084	4	17
21	3.8	0.002	0.0056	0.0036	0.0000 to 0.0071	4	14
22	3.5	0.0038	0.005	0.0013	−0.0026 to 0.0049	1	4
23	3.3	0.0046	0.0044	−0.0002	−0.0039 to 0.0033	0	−1
24	3.2	0.0036	0.0048	0.0013	−0.0025 to 0.0049	1	4
25	3.0	0.0032	0.0061	0.0028	−0.0009 to 0.0065	3	8
26	2.8	0.0033	0.0068	0.0035	−0.0003 to 0.0073	4	10
27	2.7	0.0028	0.0063	0.0035	−0.0003 to 0.0071	3	9
28	2.6	0.0022	0.0056	0.0034	−0.0007 to 0.0073	3	9
29	2.4	0.0026	0.0032	0.0006	−0.0031 to 0.0041	1	1
30	2.3	0.0032	0.0037	0.0006	−0.0030 to 0.0041	1	1

Discussion

This paper has illuminated the quality of decisions based on a biomarker risk score derived from one of the largest prospective population studies with a total of 10,263 subjects to date. The biomarkers represent three different biological pathways of CVD and their combination provides incremental value above individual risk markers and beyond classic risk factors. Studies of new biomarkers should report a variety of statistical methods and decision analytical models like the decision curve should be used when a prediction model shows potential for clinical decisions.¹⁵ On comparing the decision curve results to net reclassification,¹⁹ the results largely complement each other in that reclassifications are in the same direction, but they differ in a number of ways. The distribution of biomarker values in prospective populations can overlap substantially even when case−non-case group means are significantly different. This creates a high proportion of false positives and false negatives around any given cut-off point and the decision curve provides the best means of visualizing this. In this way, the decision curve evaluates the net benefit (incorporating true and false positives) for individual threshold points while the net reclassification evaluates benefit in categories using fixed risk cut offs. In addition, the net reclassification method considers false positives/negatives of equal value regardless of direction while the decision curve can distinguish these effects.

The decision curve approach shows that the biomarker risk score can improve CVD risk prediction at intermediate risk levels when compared to a classic risk factor model for middle-aged men in PRIME Belfast. The greatest net benefit was observed at intermediate thresholds between 10⊟20%, albeit with wide confidence intervals. In contrast, for a general population like subjects in FINRISK, with a wider age range including younger and older men and women, the biomarker risk score improves decisions in the high-risk range but not in the intermediate group. Other large population studies with broad age ranges evaluating multiple biomarkers have also suggested a limited value in this group.^11,31 Often the features of the cohort, choice of biomarker, and statistical metrics can contribute to variation in findings.^13,32 The decision curve presents the results for reclassification of the FINRISK cohort, as suggested by de Lemos and Rohatgi,³³ which highlights the complicated influence of age and sex on the interpretation of results. In this cohort, events are concentrated in the older age group and age has the largest effect on discrimination. With relatively fewer cases in the intermediate risk group, the ability to assess the potential added benefit of the biomarkers in this category is limited. The ability to test the predictive performance of the model is enhanced by the narrow age range of PRIME Belfast which deals with the dominant effect of age while providing clinically useful information on middle-aged men which explains the modest improvement in risk estimation. However, the smaller sample size of this cohort is reflected in the wider confidence intervals. In part, the confidence intervals also reflect the relative insensitivity of threshold-based measures in general, a criticism commonly associated with the c-index.³⁴ Other biomarker risk scores tested on narrow age ranges have shown improved prediction for male subgroups.³⁵ We must also bear in mind that Finland and UK have high absolute risks of CVD so the contribution of biomarkers may be specific to such populations. Individuals on cholesterol medication at baseline were removed from the analysis but some subjects will receive treatment during follow up which may influence risk estimates. Nevertheless, the decision curve was estimated with correction for optimism by assessing the confidence interval around the net benefit values, accounting for the censored nature of the data using a Kaplan Meier approach, and considering competing risk of other (non-CVD) mortality in survival analysis.

The modest statistical improvement in middle-aged men could translate to improved clinical decision making and the score could be relatively easily incorporated into the workflow of cardiovascular risk evaluation. First, the biomarkers have shown strong and consistent associations with CVD in multiple studies.^11,31,35 They can be robustly and reproducibly measured at reasonable costs and show good discrimination and calibration in our prediction model.¹⁹ While the overall clinical benefit of the model may appear modest if each threshold point is considered separately, e.g. 6 more true cases predicted per 1000 patients and 5% net fewer unnecessary treatments at a p_t= 10% for PRIME Belfast. Nevertheless, the biomarker score shows considerable clinical potential to improve risk prediction for middle-aged men at intermediate risk of CVD and should be offered to this subgroup rather than all in the population. For example, this group may include individuals with only single abnormalities of traditional risk factors (such as those with a premature family history of CVD or isolated hypertension) placing them near the threshold for deciding therapy. For these people who are undecided about starting medication for primary prevention, markedly high biomarker levels may tilt the balance in favour of treatment. By varying the threshold probability over an appropriate range, the clinician and the patient can better judge the trade-offs implicit in acting on the results of the biomarker test.

Lowering elevated levels of low-density lipoprotein cholesterol is still the primary target for reduction of CVD risk, usually with statin therapy. The JUPITER trial suggested that statin treatment may be beneficial for individuals with intermediate CVD risk, with normal cholesterol levels and raised CRP levels.³⁶ While the magnitude of this effect may be questioned³⁷ and CRP may be a marker not a causal factor for cardiovascular events,³⁸ it can still be useful for prediction, and, in addition to incorporating it in our score, CRP is one of four novel biomarkers that have been linked to the prediction of other significant outcomes such as diabetes.³⁹ Across the majority of risk thresholds, the biomarker score mainly down-classified risk for individuals without events in PRIME, suggesting its main benefit could be to reduce unnecessary statin treatments in middle-aged men. While the safety and effectiveness of deferring treatment is unknown, statins for primary prevention confer benefits across a broad range of baseline cardiovascular risk levels. Some have suggested that initiating statin treatment at lower risk levels than currently set would be cost effective provided harms associated with statins are at low levels.⁴⁰ However, treating unselected individuals with normal biomarker levels may not be practical and even patients with high risk of CVD are reluctant to begin statin therapy with 50% who begin therapy failing to adhere to treatment a year after prescription.² Statins have been linked to low intensity muscle soreness, diabetes mellitus,⁴¹ and potentially fatal liver failure or cancer.⁴² Within a narrow range of increased risk of these adverse events, biomarker screening could provide a more effective means of guiding treatment decisions as indicated by a recent cost-effectiveness study of CRP.⁴⁰ Indeed cost effectiveness studies can be more sensitive to other contextual factors, such as baseline risk of myocardial infarction and effectiveness and cost of statins or other therapy, than the actual cost of the biomarker tests, and even if these scores are more expensive than classic risk factors, their value may still be proven if clinical outcomes are sufficiently improved in terms of quality-adjusted life years.⁴³

The effectiveness of biomarkers in a clinical setting would be enhanced if biomarker guided therapies or monitoring could change cardiovascular outcomes or improve the outcome of care. A reduction in hsCRP with statins and an association between achieved concentrations of hsCRP and outcome have suggested that hsCRP could monitor response to statin therapy.⁴⁴ hsCRP may act as a treatment-effect modifier, possibly detecting nonresponsiveness to statin therapy which could spare people from unnecessary treatment.^45,46 Overall, the biomarker score could result in significant resource implications and requires evaluation in biomarker-guided randomized control trials.⁴⁰ While the decision curve approach highlights the score’s potential clinical utility in a single summary measure and adds to its assessment following AHA guidelines,¹⁷ a full decision analysis is needed to assess the cost effectiveness of screening at the population level, the behavioural response to risk perception with the added biomarker information, adherence to therapy and patient outcomes.

New assays capable of detecting very low levels of troponin have been shown to have even greater ability to identify incident coronary heart disease, mortality and hospitalization for heart failure in general populations with no previous signs of disease.⁴⁷ These highly sensitive troponin assays have the potential to refine our score but we will need to establish whether prediction based on a single troponin determination is similar or superior to our three marker score by screening in other MORGAM cohorts with different levels of absolute risk.

In conclusion, the decision curve shows the net benefit of a biomarker risk score is improved across intermediate and high risk thresholds compared with a classic risk factor model leading to a small but clinically useful increase in the identification of true cases and potentially a reduction in unnecessary treatments in a middle-aged European male cohort. While the score improves CVD prediction at high risk thresholds in a general cohort of men and women it may only provide marginal benefits among younger and older individuals. Refining CVD risk in middle-aged men could lead to more effective treatment planning to reduce CVD risk in this group.

Footnotes

Sites and key personnel of the contributing MORGAM Centres are given in the online supplementary material (Annex).

Acknowledgements

Abbott Diagnostics, BRAHMS AG, Diadexus and Roche Diagnostics provided test reagents for biomarker testing.

Funding

This work was supported by the Medical Research Council (grant number G0601463, ID number 80983, Biomarkers in the MORGAM populations). VS was supported by the Finnish Foundation for Cardiovascular Research and by the Finnish Academy grant number 129494.

Conflicts of interest

Dr Blankenberg has received honoraria from Abbott Diagnostics (<5,000 USD) and Roche Diagnostics (<5,000 USD) and has worked as a consultant/advisory board member for Brahms (<5,000 USD). No other conflicts of interest are declared.

References

Cook

Ridker

. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med 2009; 150: 795–8021.

Cooper

Nherera

Calvert

O’Flynn

Turnbull

Robson

. Clinical guidelines and evidence review for lipid modification: cardiovascular risk assessment and the primary and secondary prevention of cardiovascular disease. London: National Collaborating Centre for Primary Care and Royal College of General Practitioners, 2001. http://www.nice.org.uk/nicemedia/live/11982/40742/40742.pdf, (consulted June 2011).

Wilson

PWF

D’Agostino

Levy

Belanger

Silbershatz

Kannel

. Prediction of coronary heart disease using risk factor categories. Circulation 1998; 97: 1837–1847.

Assmann

Cullen

Schulte

. The Munster Heart Study (PROCAM) ⊟ results of follow up at 8 years. Eur Heart J 1998; 19: A2–A11.

Empana

Duchimetriere

Arveiler

Ferrieres

Evans

Ruidavets

. Are the Framingham and PROCAM coronary heart disease risk functions applicable to different European populations? The PRIME Study. Eur Heart J 2003; 24: 1903–1911.

Panagiotakos

Stvrinos

. Methodological issues in cardiovascular epidemiology; the risk of determining absolute risk through statistical models. Vasc Health Risk Manag 2006; 2: 309–315.

Koenig

. Update on integrated biomarkers for assessment of long-term risk of cardiovascular complications in initially healthy subjects and patients with manifest atherosclerosis. Ann Med 2009; 41: 332–343.

Romanens

Ackermann

Spence

Darioli

Rodondi

Corti

. Improvement of cardiovascular risk prediction; time to review current knowledge, debates, and fundamentals on how to assess test characteristics. Eur J Cardio Prev Rehab 2010; 17: 18–23.

Ridker

Paynter

Rifai

Gazino

Cook

. C-reactive protein and parental history improve global cardiovascular risk prediction: the Reynolds Risk Score for men. Circulation 2008; 118: 2243–2251.

10.

Folsom

Chambless

Ballantyne

Coresh

Heiss

. An assessment of incremental coronary risk prediction using C-reactive protein and other novel risk markers: the atherosclerosis risk in communities study. Arch Intern Med 2006; 166: 1368–1373.

11.

Wang

Gona

Larson

Tofler

Levy

Newton-Cheh

. Multiple biomarkers for the prediction of first major cardiovascular events and death. N Engl J Med 2006; 355(25): 2631–2639.

12.

Kim

Greenland

Rossouw

Manson

Cochrane

Lasser

. Multimarker prediction of coronary heart disease risk, the Women’s Health Initiative. J Am Coll Cardiol 2010; 55: 2080–2091.

13.

Hughes

Kee

Salomaa

. Clinical utility of multiple biomarker panels for cardiovascular disease risk prediction. Curr Cardiovasc Risk Rep 2011; 5: 165–173.

14.

Persell

Zei

Cameron

Zielinski

Lloyd-Jones

. Potential use of 10-year and lifetime coronary risk information for preventative cardiology prescribing decisions. Arch Intern Med 2010; 170: 470–476.

15.

Steyerberg

Vickers

Cook

Gerds

Gonen

Obuchowski

. Assessing the performance of prediction models, a framework for traditional and novel measures. Epidemiology 2010; 121: 128–138.

16.

Altman

Vergouwe

Royston

Moons

. Prognosis and prognostic research: validating a prognostic model. BMJ 2009; 338: 1432–1435.

17.

Hlatky

Greenland

Arnett

Ballantyne

Criqui

Elkind

. Criteria for evaluation of novel markers of cardiovascular risk, A scientific statement from the American Heart Association expert panel on subclinical atherosclerotic diseases and emerging risk factors and the stroke council. Circulation 2010; 119: 2408–2416.

18.

Vickers

Elkin

. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 2006; 26: 565–574.

19.

Blankenberg

Zeller

Saarela

Havulinna

Kee

Tunstall-Pedoe

for the MORGAM project. Contribution of thirty biomarkers to ten-year cardiovascular risk prediction in two population cohorts. Circulation 2010; 121(22): 2388–2397.

20.

Evans

Saalomaa

Kulathinal

Asplund

Cambien

Ferrario

for the MORGAM project. MORGAM (an international pooling of cardiovascular cohorts). Int J Epidemiol 2005; 34: 21–27.

21.

Kulathinal

Niemelä

Kuulasmaa

contributors from participating centres for the MORGAM project. Description of MORGAM cohorts. National Institute for Health and Welfare and the MORGAM Project investigators, 2005. http:/www.ktl.fi/publications/morgam/cohorts/index.html, URN:NBN:fi-fe20051214 (2005), (consulted May 2011).

22.

Vartianen

Laatikainen

Peltonen

Juolevi

Mannisto

Sundvall

. Thirty five year trends in cardiovascular risk factors in Finland. Int J Epidemiol 2010; 39: 504–518.

23.

Yarnell

. The PRIME study: classical risk factors do not explain the severalfold difference in risk of coronary heart disease between France and Northern Ireland. Prospective Epidemiological Study of Myocardial Infarction. QJM 1998; 91: 667–676.

24.

Rubin

. Multiple Imputation for Nonresponse in Surveys. New York: Wiley, 1987.

25.

Lunn

Thomas

Best

Spiegelhalter

. WinBUGS a Bayesian modelling framework: concepts, structure and extensibility. Statistics and Computing 2000; 10(4): 325–337.

26.

Moons

KGM

Donders

RART

Stijnen

Harrell

. Using the outcome for imputation of missing predictor values was preferred. J. Clin. Epidemiol 2006; 59: 1092–1101.

27.

Kalbfleisch

Prentice

Kalbfleisch

Prentice

. Competing risks and multistate models. The Statistical Analysis of Failure Time Data, 2nd ed. New York: Wiley, 2002, pp. 252–252.

28.

Vickers

Cronin

Elkin

Gonen

. Extension to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers. Med Inform Decis Making 2008; 53(8): 1–17.

29.

Steyerberg

Harrell

Jr Borsboom

Eijkemas

Habbema

. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001; 54: 774–781.

30.

R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2010. http://www.r-project.org (2010, consulted October 2010).

31.

Melander

Newton-Cheh

Almgren

Hedblad

Berglund

Engstrom

. Novel and conventional biomarkers for the prediction of incident cardiovascular events in the community. JAMA 2009; 302(1): 49–57.

32.

Wang

. Multiple biomarkers for predicting cardiovascular events, lessons learned (editorial comment). J. Am Col Cardiol 2010; 55(19): 2092–2095.

33.

de Lemos

Rohatgi

. Separating the contenders from the pretenders, competitive high throughput biomarker screening in large population-based studies (editorial). Circulation 2010; 121(22): 2381–2383.

34.

Cook

. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007; 115: 928–935.

35.

Zethelius

Berglund

Sundström

Ingelsson

Basu

Larsson

. Use of multiple biomarkers to improve the prediction of death from cardiovascular causes. N Engl J Med 2008; 358(20): 2107–2116.

36.

Ridker

Danielson

Fonseca

FAH

Genest

Gotto

Kastelein

JJP

. Rosuvastatin to prevent vascular events in men and women with elevated C-reactive protein. N Engl J Med 2008; 359: 2195–2207.

37.

de Lorgeril

Salen

Abramson

Dodin

Hamazaki

Kostucki

. Cholesterol lowering, cardiovascular diseases, and the Rosuvastatin-JUPITER controversy, a critical appraisal. Arch Intern Med 2010; 170(12): 1032–1036.

38.

The Emerging Risk Factors Collaboration. C-reactive protein concentration and risk of coronary heart disease, stroke, and mortality: an individual participant meta-analysis. Lancet 2010; 375: 132–140.

39.

Salomaa

Havulinna

Saarela

Zeller

Jousilahti

Jula

. Thirty-one novel biomarkers as predictors for clinically incident diabetes. PloS One 2010; 5: 1–8. e10100.

40.

Lee

Cipriano

Owens

Hlatky

. Cost effectiveness of using high sensitivity C-reactive protein to identify intermediate and low cardiovascular risk individuals for statin therapy. Circulation 2010; 122: 1478–1487.

41.

Saatar

Preiss

Murray

Welsh

Buckley

de Craen

AJM

. Statins and risk of incident diabetes: a collaborative meta-analysis of randomised statin trials. Lancet 2010; 373: 735–742.

42.

Karp

Behlouli

Lelorier

Pilote

. Statins and cancer risk. Am J Med 2008; 121: 302–309.

43.

Schnell-Inderst

Schwarzer

Gohler

Grandi

Grobein

Stollenwerk

. Prognostic value, clinical effectiveness, and cost-effectiveness of high sensitivity C-reactive protein as a marker for major cardiac events in asymptomatic individuals: A health technology assessment report. Int J Tech Ass Health Care 2010; 26: 30–39.

44.

Ridker

Cannon

Morrow

Rifai

Rose

McCabe

. C-reactive protein levels and outcomes after statin therapy. N Eng J Med 2005; 352: 20–28.

45.

Ridker

Rifai

Clearfield

Downs

Weis

Miles

. Measurement of C-reactive protein for the targeting of statin therapy in the primary prevention of acute coronary events. N Eng J. Med 2001; 344: 1959–1965.

46.

McMurray

Kjekshus

Gullestad

Dunselman

Hjmarson

Wedel

. Effects of statin therapy according to plasma high sensitivity C-reactive protein concentration in the Controlled Rosuvastatin Mutlinational Trial in Heart Failure (CORONA); a retrospective analysis. Circulation 2009; 120: 2188–2196.

47.

Saunders

Nambi

de Lemos

Chambless

Virani

Boerwinkle

. Cardiac troponin T measured by a highly sensitive assay predicts coronary heart disease, heart failure, and mortality in the Atherosclerosis Risk in the Communities Study. Circulation 2011; 123: 1367–1376.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB

0.04 MB