Beyond the numbers: a critique of quantitative multi-criteria decision analysis

When setting priorities for health, there is broad agreement that a range of social values and ethical principles beyond clinical and cost-effectiveness matter, but exactly how health technology assessment (HTA) should account for a broader set of criteria remains an area of ongoing debate. In light of this, we welcome a recent review paper by Baltussen et al. evaluating the potential of different multi-criteria decision analysis (MCDA) approaches to enable HTA agencies to incorporate a broader set of values in their appraisals. The authors describe three approaches to MCDA-qualitative MCDA, quantitative MCDA, and MCDA with decision rules-laying out their relative advantages and disadvantages and providing recommendations for how they can best be implemented. While we endorse many of the authors' assessments and conclusions, including the critical role of deliberation in any MCDA approach and the undertaking of qualitative MCDA at a minimum, we take a stronger position regarding the flaws of quantitative MCDA and strongly caution against it. We find quantitative MCDA antithetical to at least two of the ways MCDA is intended to improve HTA recommendations: (i) enhancing quality and (ii) promoting transparency. Quantitative MCDA may mask the complex tradeoffs that exist within and between decision criteria and remain generally inaccessible to those who are not well-versed in its technical methods of appraisal. We advocate for a predominantly qualitative approach to MCDA appraisal centered around deliberation and supplemented with decision aids to help account for health opportunity costs.