Treatment Effect in Earlier Trials of Patients With Chronic Medical Conditions: A Meta-Epidemiologic Study

To determine whether the early trials in chronic medical conditions demonstrate an effect size that is larger than that in subsequent trials.

We identified randomized controlled trials (RCTs) evaluating a drug or device in patients with chronic medical conditions through meta-analyses (MAs) published between January 1, 2007, and June 23, 2015, in the 10 general medical journals with highest impact factor. We estimated the prevalence of having the largest effect size or heterogeneity in the first 2 published trials. We evaluated the association of the exaggerated early effect with several a priori hypothesized explanatory variables.

We included 70 MAs that had included a total of 930 trials (average of 13 [range, 5-48] RCTs per MA) with average follow-up of 24 (range, 1-168) months. The prevalence of the exaggerated early effect (ie, proportion of MAs with largest effect or heterogeneity in the first 2 trials) was 37%. These early trials had an effect size that was on average 2.67 times larger than the overall pooled effect size (ratio of relative effects, 2.67; 95% CI, 2.12-3.37). The presence of exaggerated effect was not significantly associated with trial size; number of events; length of follow-up; intervention duration; number of study sites; inpatient versus outpatient setting; funding source; stopping a trial early; adequacy of random sequence generation, allocation concealment, or blinding; loss to follow-up or the test for publication bias.

Trials evaluating treatments of chronic medical conditions published early in the chain of evidence commonly demonstrate an exaggerated treatment effect compared with subsequent trials. At the present time, this phenomenon remains unpredictable. Considering the increasing morbidity and mortality of chronic medical conditions, decision makers should act on early evidence with caution.