Trends in Author-Reported Cost-Effectiveness Thresholds in the United States from 1995 to 2018: Implications for Discount Rates

Introduction

The First and Second Panels on Cost-Effectiveness in Health and Medicine recommend using equal real (i.e., inflation-adjusted) discount rates for health and cost outcomes (set at 3% annually for US-based analyses) in cost-effectiveness analyses (CEAs).^1,2 Decisions based on CEA using equal discount rates would be consistent with using constant cost-effectiveness thresholds over time and avoid the Keeler-Cretin paradox.^3,4

Cost-effectiveness thresholds can be based alternatively on the marginal consumption value of health (v) or the marginal health productivity of a health care system (k). However, health spending trends suggest cost-effectiveness thresholds could be increasing over time.^5,6 Changes in thresholds over time can result from changes in the consumption value of health, marginal productivity, or both. Increasing average wealth over time would support an increasing consumption value of health, assuming health care is a normal good. However, it is possible that supply-side cost-effectiveness thresholds for health could decrease or remain stable over time despite an increasing consumption value. For example, if more cost-effective interventions are introduced over time (or if existing options become more cost-effective, from lower prices for generic medications, for example), then a shadow price-based cost-effectiveness threshold (i.e., where the last dollar is spent in a conventional “shopping spree” problem) could decrease over time even as the budget for health spending increases.^7,8 Even if the real cost-effectiveness threshold is constant over time, the nominal cost-effectiveness threshold will increase with inflation. Increasing or decreasing real cost-effectiveness thresholds over time would imply discount rates used for health and cost outcomes should not be equal.^6,9,10

The United States has never had a government-endorsed cost-effectiveness threshold or widely used empirical estimate of the opportunity costs of health spending. Instead, authors of CEAs performed for the US setting have typically used cost-effectiveness thresholds citing values used in previous studies or fieldwide conventions. Despite the lack of a formal US threshold, most authors use cost-effectiveness thresholds to interpret their cost-effectiveness results (85% of CEAs performed for the US setting referenced a cost-effectiveness threshold in 2010–2012, compared with less than 50% in 1990–1999). ¹¹ The authors’ choice of threshold can offer insights into what the research community believes is the relevant threshold of US health policy.

Just as a constant cost-effectiveness threshold is consistent with discounting health and costs at the same rate, so a threshold that changes over time is consistent with discounting health and costs at different rates. We use this relationship to estimate the difference between discount rates for health and costs that is consistent with the observed increase in cost-effectiveness thresholds. Specifically, we sought to 1) visually depict the relationship between cost-effectiveness thresholds over time and discount rates and 2) retrospectively assess whether the recommended equal discount rates for costs and health is consistent with author-reported cost-per quality-adjusted life-year (QALY) thresholds from CEAs published for the US setting over 24 y.

Methods

Relationship between Cost-Effectiveness Thresholds over Time and Discount Rates

Consider a simple example. A program imposes costs c_t and provides health benefits h_t in period t. In period t, a decision maker will be indifferent to funding or not funding the program if and only if its period t incremental cost-effectiveness ratio (ICER) c_t/h_t = λ _t , where λ _t is the cost-effectiveness threshold in period t. In period 0, a decision maker will be indifferent to funding the program if and only if its period 0 ICER (the ratio of the present value of costs to the present value of health effects) c t ( 1 + d c ) t / h t ( 1 + d h ) t = λ ₀ , where λ ₀ is the cost-effectiveness threshold in period 0 and d_c and d_h are the discount rates for costs and health, respectively. The decisions at time 0 and time t will be consistent if and only if

λ t = ( 1 + d h ) t ( 1 + d c ) t λ 0 .

(1)

Equation (1) shows how the discount rates for health and for costs and the rate at which the cost-effectiveness threshold changes over time are logically related, if decisions at different times are consistent. The relationship is illustrated in Figure 1. If the decision maker uses equal discount rates for costs and health (d_c = d_h), then the ICER for implementing the program in time period t (c_t/h_t) will be equal to the ICER for implementing program in time period 0. Alternatively, if the cost-effectiveness threshold changes over time, then health and costs must be discounted at different rates. Hammitt ¹⁰ previously showed that if we know the values of any 2 terms (among d_c, d_h, λ _t /λ₀,) we can solve for the third. Specifically, if we know the discount rate for costs and how the cost-effectiveness threshold changes over time, we can rearrange equation (1) to solve for the discount rate for health,

d h = ( 1 + d c ) ( λ 0 λ t ) 1 / t − 1 .

(2)

OPEN IN VIEWER

Figure 1

Relationship between costs and health outcomes across 2 time periods. The exchange rate between incremental costs in time period 0 (c₀) and incremental costs in time period t (c_t) is the discount rate for costs (d_c). The exchange rate between incremental health benefits in time period 0 (h₀) and incremental health benefits in time period t (h_t) is the discount rate for health (d_h). The exchange rates between incremental costs and incremental health are the cost-effectiveness thresholds (“ce threshold” in the figure) for health values in time periods 0 and t (λ₀ and λ _t , respectively). If d_c = d_h, then λ₀ = λ _t .

Results

We found 1995 CEAs published for the US setting using a year of currency between 1995 and 2018. Among the CEAs, the average difference between publication year (which ranged from 1996 to 2019) and currency year was 2.33. The nominal author-reported cost-effectiveness thresholds ranged from $5,000/QALY to $1,000,000/QALY, with a mean of $73,565/QALY (Table 1). The median nominal threshold was $50,000/QALY for each year from 1995 through 2005, between $50,000/QALY and $75,000/QALY for 2006 to 2012, $75,000/QALY for 2013 to 2015, and $100,000/QALY for 2015 to 2018. Another commonly reported threshold was $29,300/QALY (by 25 CEAs, all published before 2010), referring to a cost-effectiveness threshold used in a landmark paper by Laupacis et al. ¹⁶ in 1992. After adjusting for inflation, the mean cost-effectiveness threshold across these 1995 CEAs was $52,405/QALY (1995$). When 5 outlier cost-effectiveness thresholds (4 at greater than $300,000/QALY and 1 at less than $10,000/QALY nominal) were excluded (Supplementary Table A-1), these averages were $71,988/QALY (nominal) and $51,319/QALY (inflation adjusted, 1995$).

The average nominal author-reported cost-effectiveness threshold increased over time (Figure 2), from a mean of $57,620/QALY in 1995 to $95,213/QALY in 2018 (year slope coefficient = 1954, P < 0.001, adjusted R² = 0.05092), with substantial variance in each year (Figure 3). After adjusting for inflation, the author-reported cost-effectiveness thresholds were relatively more stable over time (Figure 2), with a mean of $57,786/QALY in 2018 (converted to 1995$). Using the linear trend data over the 24-y time period (1995–2018; year slope coefficient = 300.9, P = 0.0215, adjusted R² = 0.002148), and fixing the annual discount rate for costs at 3%, the calculated discount rate for health would be 2.40% (95% CI 2.04%–2.78%); the same calculated discount rate for health would be 2.46% (2.08%–2.79%) when excluding the 5 outlier author-reported cost-effectiveness thresholds (Supplementary Figure A-1) and 2.30% (1.93%–2.70%) when using the geometric mean for CEAs in which authors reported more than 1 cost-effectiveness threshold. Most (98.9%) CEAs reported equal d_h and d_c values, with 93.7% of the CEAs we analyzed using an annual discount rate of 3% for both. All CEAs that reported unequal discount rates (1.1% of all the CEAs) used lower values for d_h than for d_c.

OPEN IN VIEWER

Figure 2

Mean nominal author-reported cost-effectiveness thresholds (“c-e thr” in the figure) in cost-effectiveness analyses (CEAs) published for the US setting increased from 1995-2018 (n=1,995 CEAs). This trend was relatively more flat after adjusting for inflation (“Adj c-e thr” in the figure).

OPEN IN VIEWER

Figure 3

Box plot of nominal author-reported cost-effectiveness thresholds over time (excluding 5 outlier cost-effectiveness thresholds >$300,000/quality-adjusted life-year or <$10,000/quality-adjusted life-year).

We noticed that inflation-adjusted author-reported cost-effectiveness thresholds increased with a higher slope during currency years 2015 to 2018. When we restricted our primary analysis to currency years 1995 to 2014, we found that the calculated discount rate for health would be 2.99% (2.37%−3.45%). When we restricted our primary analysis to currency years 2015 to 2018, we found that the calculated discount rate for health would be −2.91% (−6.55% to 1.18%). A negative discount rate for health would imply that decision makers would be willing to trade 1 QALY at time = 0 for less than 1 QALY at time = 1 (i.e., the value of health increases over time). We also noticed that 1995 was an outlier in our data; when we restricted our analysis to currency years 1996 to 2018, we found that the calculated discount rate for health would be 2.30% (1.90%−2.69%). When we limited our analysis to CEAs that used a lifetime analytic time horizon (n = 968), we found that the calculated discount rate for health would be 2.32% (1.88%–2.82%).

Discussion

We analyzed author-reported cost-effectiveness thresholds used in CEAs published for the US setting with currency years between 1995 and 2018 and found that average nominal and inflation-adjusted cost-effectiveness thresholds increased over that period. Assuming an annual discount rate for costs equal to 3%, the discount rate for health would need to equal 2.43% to 2.48% (depending on the subset of CEAs analyzed) to preserve consistency across decisions taken at different dates given the observed trends in real author-reported cost-effectiveness thresholds. We also found that restricting our analysis to currency years 1995 to 2014 would result in an implied discount for health of 2.99%, essentially equal to a discount rate of 3%. Trend data in recent (i.e., post-2014) years showed a shift toward CEA authors using higher cost-effectiveness thresholds for US-based analyses, although this time period was relatively short and contained only 352 of the 1995 total CEAs in our data set. ¹²

The implication of the stable real average cost-effectiveness thresholds over the restricted time period of 1995 to 2014 suggests that cost-effectiveness researchers, on average as a field, chose cost-effectiveness thresholds that were implicitly consistent with expert recommendations to use equal discount rates for cost and health outcomes for US-based CEAs during this time period. After 2014, however, CEA researchers started to use higher cost-effectiveness thresholds in a manner that would imply that a lower discount rate for health would be needed to preserve consistency across decisions taken at different times. It is possible that the 2014 New England Journal of Medicine Perspectives article by Neumann et al. ¹¹ (cited more than 1500 times as of June 2021), which suggested CEA researchers use a $100,000/QALY or $150,000/QALY cost-effectiveness threshold for the United States instead of the prevailing threshold of $50,000/QALY, accelerated a shift toward the higher thresholds. One practical recommendation in light of our analysis could be for expert panels to release recommended cost-effectiveness thresholds on a more regular basis (e.g., updated every 3 or 5 y).

In addition to the 1996 and 2016 US panels on Cost-Effectiveness in Health and Medicine, most other national guidelines also recommend using equal discount rates for costs and health outcomes. A 2018 review of discounting by Attema et al. ¹⁷ found that only 4 countries (Belgium, the Netherlands, Poland, and Russia) of the 21 they analyzed recommend a different (lower, in each case) discount rate for health outcomes than for costs in their national guidelines for economic evaluation. There is also evidence of conflicting recommendations across subfields within economic evaluation. In the United States, Department of Health and Human Services guidance on regulatory impact analysis includes increasing values of statistical life over time based on changes in real income growth, which implies increasing inflation-adjusted cost-effectiveness thresholds over time and thus a lower discount rate for health than for costs. ¹⁸ We did not analyze author-reported cost-effectiveness thresholds for settings aside from the United States.

We found that almost all (98.9%) individual CEA studies used equal discount rates for costs and health, implying consistency within each of these article. Our data from currency years 1995 to 2018 show that the field, however, has been inconsistent, recommending increasing thresholds over time that imply health increments should be discounted at a smaller rate than costs. Another recommendation, based on this apparent discrepancy between what authors implicitly assume within and across CEA article, would be for the next Panel on CEA in Health and Medicine to further examine whether discount rates should be equal for health and cost outcomes. In a recent survey of health economists who performed at least 1 CEA, only 29% of the 123 researchers believed their choice of cost-effectiveness threshold represented either the true consumption value of health or the opportunity cost of health spending (or both); 61% of respondents chose their threshold solely based on convention. ¹⁴ We argue here that these conventions would be more appropriate if they were consistent with the underlying theory that connects discount rates and trends in the value of health.

Despite this argument, the analysis we present in this article is descriptive, not normative. Braithwaite and Roberts ¹⁹ offered a standard gamble–based approach to estimate societal preferences for health outcomes over time, which could be used to derive discount rates that should be used by CEA researchers. O’Mahony et al. ²⁰ discussed, among other issues, the political realities of the National Institute of Health and Care Excellence 2020 methods review of discounting used in health technology assessments for the United Kingdom, such as how lowering the discount rate from 3.5% to 1.5% (for both cost and health outcomes) would affect the ICERs of some interventions more than others (such as vaccination programs with high upfront costs and delayed health benefits). Paulden et al. ²¹ presented an argument that 3% might be too high to use for a discount rate for costs in the United States based on recent data on real interest rates on US treasury notes and bonds (that vary between 0.3% and 1.5% per annum). Our analysis depends on the relative differences in discount rates, however, and could be adjusted for any discount rate value for costs; a 1% discount rate for costs would be consistent with a 0.4% discount rate for health using our other base-case assumptions. In addition, we assumed fixed discount rates in our analysis, despite empirical evidence that shows that individuals commonly use hyperbolic discount rates. ²² We do not believe, however, that such evidence provides a normative basis for policy makers to use hyperbolic discounting, and the lack of such a basis could explain why fixed discount rates are common assumptions in economic evaluations.

Any change to the recommended discount rate for health outcomes should reflect societal preferences and practical realities, as these articles suggest. ²³ Our article adds a theory-based visualization (Figure 1) for how the choice of discount rates interacts with changes in cost-effectiveness thresholds over time, in addition to quantifying what CEA researchers have implicitly assumed for US analyses from 1995 to 2018. More empirical evidence on the opportunity costs of health spending or direct estimates of willingness-to-pay for QALYs gained for the United States, and how these estimates have changed over time, could help establish trends in willingness-to-pay for QALYs that could be used to infer a discount rate for health that is consistent with these trends. ¹⁵

The relationship between incremental costs, incremental health, cost-effectiveness thresholds, and discount rates can be more nuanced than the visual framework we present in Figure 1. As Claxton et al. and Paulden and Claxton have noted, this relationship depends on a number of factors, including but not limited to whether health budgets are fixed (implying a supply-side cost-effectiveness threshold, k_h, should be used) or not (implying a demand-side cost-effectiveness threshold, v, should be used), whether the consumption value of health changes over time, whether the decision maker’s goal is to maximize welfare or population health itself, and the rate at which health payers can borrow between time periods.^4,24,25 Future work can extend our visual depiction for when the opportunity costs of foregone health do not equal the consumption value of health (i.e., when k_h≠v).

In summary, our retrospective analysis of author-reported cost-effectiveness thresholds for US-based CEAs suggests that researchers in the field have implicitly assumed that inflation-adjusted cost-effectiveness thresholds in the United States have been increasing over time, which is inconsistent with the recommended and prevailing choice of equal discount rates for health and cost outcomes. This finding depends on the cutoff year used in the analysis (data from currency years 1995–2014 would support use of equal discount rates, whereas data after 2014 would suggest a sharper trend toward increasing cost-effectiveness thresholds), however. Future recommendations for CEA should be based on additional research that assesses whether the use of constant inflation-adjusted cost-effectiveness thresholds (implied by equal discount rates for costs and health) moving forward is justified by either societal time preferences for cost and health outcomes or empirical trends in the opportunity costs of health spending or the societal value of a QALY.

Supplemental Material