A Note on the Expected Biases in Conventional Iterative Health State Valuation Protocols

Hypothetical bias is the difference between hypothetical (stated) preference and actual choice behavior. Methods have been employed in the CV literature to mitigate this, such as “cheap talk.” Here, the potential for hypothetical bias is discussed with respondents prior to eliciting their WTP,^3,4 but arguably this could result in a counterbias. Health state valuations involve hypothetical decisions and thus are susceptible to hypothetical bias. However, the degree of the bias is likely to vary: some respondents are in full health, so that the health state being valued is hypothetical, but not the full health; other respondents may be in a health state similar to the one being valued, so that full health is hypothetical; the durations of TTO scenarios are usually hypothetical and more so for younger respondents; and the very task of trading duration off for quality of life is entirely hypothetical. Furthermore, unlike CV studies, there is no obvious way of establishing the extent of this bias in health state valuation.

Scenario misspecification bias is where the scenario presented to respondents is misunderstood. ⁵ The recommended MVH TTO protocol values a given health state lasting for 10 years, followed by death. This is done by identifying the number of years to live in full health followed by death that is equivalent to it. Although people have everyday experiences of transactions involving money, the kind of questions asked in health state valuation exercises is completely idiosyncratic. They may therefore be more susceptible to scenario misspecification.

Payment vehicle bias occurs when different payment vehicles, such as out-of-pocket payment, payment via insurance premium, and payment via taxation, influence WTP. ⁶ Payment mode bias is where the mode of payment, such as simply stating raw WTP or WTP as a percentage of income, can influence final values. Because CV bids are presented in terms of amounts of money to be paid to have the good (bids), the parallel concept in TTO would be numbers of years to be given up to live in full health. However, most TTOs are presented in terms of numbers of years to be lived in full health (let us refer to these as “offers”). The effect of payment vehicle bias under such framing is unknown. Furthermore, because the TTO does not usually specify how the individual would come to die in each alternative, it is possible that payment vehicle/mode biases are less relevant.

Question order bias can arise when more than one good is valued and the value of a good is dependent on the order in which that good was presented to respondents. In CV, the first good valued tends to be given the highest WTP. There is limited and mixed evidence of question ordering effects in health using contingent valuation methods. ⁷ Health state valuation studies typically ask respondents to value a number of states, which makes them susceptible to question order bias. It has been observed that the TTO value of a health state is affected by the number of health states preceding it in the valuation exercise. ⁸ The typical way of neutralizing question order bias in the aggregate is to randomize the order in which different health states are valued.

Range bias exists when WTP is influenced by the range of values presented to respondents. This has been found in CV in health. ⁹ The use of visual aids in TTO may invite range bias because the TTO “board” visualizes the range of values that are available to the respondent. Since the upper bound for positive values indicated by the TTO board (1.0) is the theoretically determined upper bound, the impact this may have on the TTO values is much less than the range bias in the CV, where the choice of upper bound is (in most cases) entirely arbitrary. On the other hand, range bias may affect the lower bound of positive values and the upper bound of negative values, because that is where the visual aid is bound, although values are not. However, range bias introduced at the lower bound of negative values is possibly the most serious. The MVH protocol involves transforming negative values after elicitation; the elicited negative values using the TTO board can range from −39 to 0, but it is not clear whether respondents fully appreciate this or how they would react if they did. The recently developed “lead-time” version of TTO, which reconciles the inconsistency between the visual aid and implied values in the negative range, ¹⁰ is unlikely to be affected by range bias around zero but suffers from its own range bias, and the values are affected by the lowest value that the visual aid accommodates. ¹¹ In addition, if TTO studies use different time horizons, range bias may compromise the comparability of results across studies.

Yea-saying (also known as acquiescence bias) occurs if respondents reply “yes” when they are asked whether they are willing to pay an amount regardless of whether they are actually willing to pay it.^12,13 Yea-saying could occur in all close-ended methods of eliciting WTP and would lead to artificially high estimates of mean WTP. As long as a TTO respondent is asked which of the 2 possible lives he or she prefers, yea-saying is likely to be avoided, but yea-saying may be relevant if the respondent is asked whether he or she would give up years of life in exchange for better health.

Starting point bias (or anchoring) exists when individuals anchor their WTP to the initial bid, resulting in a correlation between the starting bid and the final monetary amount. Starting point bias is a well-documented problem in the CV literature.^13–19 The MVH TTO protocol for the first state evaluated begins with an offer at 1.0. Typically, respondents will take the offer. The second step is an offer at 0, and the third step is an offer at 0.5. The recommended MVH protocol skips the first step for the second health state onwards and begins with an offer at 0. So if there is a starting point bias, this may depend on the position of the health state in the survey. In addition, there may be a starting point bias within each step, subject to the first offer within the step. There is some evidence based on a large general public survey that when the initial offer is varied, the final TTO values vary in line with it, including whether the health state is better or worse than being dead. ²⁰

Incentive incompatibility implies the iterative nature of the elicitation method may provide the incentive to behave in a strategic fashion in follow-up questions, such that responses are not a true measure of WTP ¹⁸ ; this has been observed in health CV studies. ²¹ The bias would arise in TTO studies if respondents wish to secure an offer that is higher than what is equivalent to their valuation of the health state, so that TTO values became upward biased. However, because nobody has direct experience of markets where a longer survival in poorer health could be exchanged for a shorter survival in better health through incentive compatible procedures, it is not clear how strategically people would behave when faced with such tradeoffs. Furthermore, there may be other possible strategic behaviors. If, for example, people believed (rightly or wrongly) that patients would benefit if health problems were exaggerated, then this may lead to strategic behavior in the opposite direction.