Appropriate use of information in therapeutic decision-making: reflections on indirect comparisons

Although the statistical strength of direct comparative randomized controlled trials is generally acknowledged, the particular demands of therapeutic decision making will often require indirect comparisons to be made, based on pooled data from multiple trials. As for all post-hoc analyses, the process of indirect comparison runs the risk of introducing significant bias into the results and consequently a robust statistical approach is required, in order to minimise the risk. To address this problem, a range of different methodologies have been developed over the past twenty years, using both frequentist and Bayesian models. It is important to appreciate the strengths and limitations of these techniques: however, the technical complexities tend to make this type of analysis somewhat opaque to the non-specialist reader. In this article, we consider the use of a simple, non-specialist critical appraisal tool developed by ISPOR, which allows methodological and interpretive errors to be identified and flagged as potential sources of bias, even when the detailed statistical methodology is not well understood by the reader.