Overestimation of benefit when clinical trials stop early: a simulation study

Area of Science:

• Clinical Trials
• Biostatistics
• Medical Research Methodology

Background:

• Early stopping of clinical trials based on favorable interim analyses can lead to exaggerated estimates of treatment benefit.
• This study quantifies the extent of benefit overestimation in trials stopped early for efficacy.

Purpose of the Study:

• To simulate clinical trials that stop early for benefit and estimate the degree of benefit overestimation.
• To compare benefit overestimation in typical real-world truncated trials versus those continuing to completion and large SPRINT-like trials.

Main Methods:

• One million trials were simulated, with parameters like observation period, subject number, and control event rate randomized.
• Intervention effects varied, including warfarin-like (60% RRR), statin-like (35% RRR), ASA-like (15% RRR), no effect (0% RRR), and harm (20% RRI).
• Trials used a single interim analysis with a z-value of 2.782 (O'Brien-Fleming threshold); large SPRINT-like trials and non-interrupted trials were also modeled.

Main Results:

• Typical truncated trials showed a true relative risk reduction (RRR) of approximately 2/3 of the observed RRR.
• Absolute RRR was overestimated by a median of 14.9% in typical truncated trials, 5.3% in trials to completion, and 2.3% in SPRINT-like trials.
• A minimum of 250 events were required across all models to limit absolute RRR overestimation to below 5%.

Conclusions:

• Clinical trials stopped early for benefit typically overestimate the true relative risk reduction by about 50%.
• Benefit overestimation was significantly smaller in large SPRINT-like trials stopped early.
• Regardless of trial size or early stopping, at least 250 events are necessary to ensure absolute RRR overestimation remains below 5%.