Evolution of treatment effects over time: empirical insight from recursive cumulative metaanalyses

Evidence on how much medical interventions work may change over time. It is important to determine what fluctuations in the treatment effect reported by randomized trials and their metaanalyses may be expected and whether extreme fluctuations signal future major changes. We applied recursive cumulative metaanalysis of randomized controlled trials to evaluate the relative change in the pooled treatment effect (odds ratio) over time for 60 interventions in two medical fields (pregnancy/perinatal medicine, n = 45 interventions; myocardial infarction, n = 15 interventions). We evaluated the scatter of relative changes for different numbers of total patients in previous trials. Outlier cases were noted with changes greater than 2.5 standard deviations of the expected. With 500 accumulated patients, the pooled odds ratio may change by 0.6- to 1.7-fold in the immediate future. When 2000 patients have already been randomized, the respective figures are between 0.74- and 1.35-fold for pregnancy/perinatal medicine and between 0.83- and 1.21-fold for myocardial infarction studies. Extreme early fluctuations in the treatment effect were observed in three interventions (magnesium in myocardial infarction, calcium and antiplatelet agents for prevention of preeclampsia), where recent mega-trials have contradicted prior metaanalyses, as well as in four other examples where early large treatment effects were dissipated when more data appeared. Past experience may help quantify the uncertainty surrounding the treatment effects reported in early clinical trials and their metaanalyses. Early wide oscillations in the evolution of the treatment effect for specific interventions may sometimes signal further major changes in the future.