Impact of small study bias on cost-effectiveness acceptability curves and value of information analyses

It is well known that small, randomized, controlled trials (RCTs) have limited validity. When comparing the results of meta-analyses with those of later large trials or with those of large trials removed from the meta-analyses, discrepancies were reported. This paper addresses two issues: (1) how measures of the uncertainty in cost-effectiveness, i.e., cost-effectiveness acceptability curves (CEACs), and the expected value of perfect information (EVPI) are affected by the limited validity of small trials and (2) how to deal with this bias. To this end, the paper adopts a Bayesian approach. Using empirical estimates for the validity of small RCTs compared to larger RCTs, the probability of cost-effectiveness drops by almost 10 %, while the EVPI is three times higher. In conclusion, traditional CEACs and EVPI analyses based on (small) RCTs may need careful appraisal. Ignoring prior evidence on the validity of small-size trials leads to an underestimation of uncertainty in cost-effectiveness. For future economic analyses, it is important to incorporate aspects of uncertainty which are caused by flawed data on effectiveness .