Data and models determine treatment proposals--an illustration from meta-analysis

A relevant problem in meta-analysis concerns the possible heterogeneity between trial results. If a test of heterogeneity is not significant the trials are often considered to be "homogeneous" and the individual trial results are replaced by an overall mean effect size and its confidence interval ("equal effects model"). If the trials are heterogeneous the individual trial effect sizes are conserved ("fixed effects model"). In a more flexible approach ("random effects model"), each trial makes use of knowledge from the other trials so individual effect sizes are "shrunken" towards an overall mean effect size. The more flexible tool may be useful for doctors involved in a trial when the outcome of their individual trial differs markedly from the overall mean effect size. Where a particular trial result is opposite in direction to the overall mean result, a conflict may arise: should a new patient be treated with the new method or not? The more flexible position and a graphical comparison of the three approaches are likely to be helpful in guiding the decision. Applying different models to the same data may lead to apparently paradoxical results: an individual trial result may be interpreted to be beneficial or harmful depending on the choice of model.