Systematic reviews of economic evaluations: utility or futility?

Systematic reviews of studies of effectiveness are the centrepiece of evidence-based medicine and policy making. Increasingly, systematic reviews of economic evaluations are also an expected input into much evidence-based policy making, with some health economists even calling for 'an economics approach to systematic review'. This paper questions the value of conducting systematic reviews of economic evaluations to inform decision making in health care. It argues that the value of systematic reviews of economic evaluations is usually undermined by three things. Firstly, compared with effectiveness studies, there is a much wider range of factors that limit the generalisability of cost-effectiveness results, over time and between health systems and service settings, including the context-dependency of resource use and opportunity costs, and different decision contexts and budget constraints. Secondly, because economic evaluations are more explicitly intended to be decision-informing, the requirements for generalisability take primacy, and considerations of internal validity become more secondary. Thirdly, since one of the two main forms of economic evaluation - decision analytic modelling - is itself a well-developed method of evidence synthesis, in most cases the need for a comprehensive systematic review of previous economic evaluations of a particular health technology or policy choice is unwarranted. I conclude that apparent 'meta-analytic expectations' for clear and widely applicable cost-effectiveness conclusions from systematic reviews of economic evaluations are optimistic and generally futile. For more useful insights and knowledge from previous economic studies in evidence-based policy making, a more limited range of reasons for conducting systematic reviews of health economic studies is proposed.