The validity of QALYs: an experimental test of constant proportional tradeoff and utility independence

Pliskin, Shepard, and Weinstein identified three preference conditions that ensure that quality-adjusted life years (QALYs) represent preferences over gambles over chronic health profiles. This paper presents an experimental test of the descriptive validity of two of these preference assumptions: utility independence and constant proportional tradeoff. Eighty students at the Stockholm School of Economics and 92 students at Erasmus University Rotterdam participated in the experiment. The results of the experiment support the descriptive validity of constant proportional tradeoff: both within groups and between groups constant proportional tradeoff could not be rejected. The results are less supportive of the descriptive validity of utility independence. Within-groups utility independence was rejected. Between-groups utility independence could not be rejected, but this may have been due to a lack of statistical power. Analysis of the individual responses revealed that without adjustment for imprecision of preference, 39 respondents (22.8%) satisfied constant proportional tradeoff. Twenty-three respondents (13.4%) satisfied utility independence without adjustment for imprecision of preference. However, because of the relative unfamiliarity of the respondents with both the health states to be evaluated and the methods of health-state-utility measurement, it is likely that the respondents' preferences were imprecise. Adjusted for imprecision of preference, the upper estimates of the proportions of respondents who satisfied constant proportional tradeoff and utility independence, respectively, were 90.1% (155 respondents) and 75.6% (130 respondents). Pliskin et al. further derived that if an individual's preferences satisfy both constant proportional tradeoff and utility independence, then these preferences can be represented by a more general, risk-adjusted QALY model. Without adjustment for imprecision of preference, ten respondents (5.8%) satisfied both constant proportional tradeoff and utility independence. Adjusted for imprecision of preference, the upper estimate of the proportion of respondents who satisfied both constant proportional tradeoff and utility independence was 68.6% (118 respondents). The results of this study indicate that constant proportional tradeoff holds approximately. The evidence is much weaker for utility independence, however. This has important implications for the use of QALY-type measures in medical decision making.