Reducing overestimation of the treatment effect by interim analysis when designing clinical trials

Several researchers in the statistical and medical communities have noted the overestimation of the treatment effect when a trial is stopped early in the interim analysis for efficacy; however, methods to reduce this overestimation are rarely used because the overestimation mechanisms are not well understood by many in clinical trial practice. A trial design that leads to less overestimation is needed.

A computer simulation of hypothetical clinical trials is used to visually explain why the overestimation occurs. A quantitative evaluation of the magnitude of the overestimation is made according to the characteristics of the trial design, such as the total number of events, number of events in the interim analysis, proportion of the number of events to total events and the type of α-spending function.

When the total number of events was more than or equal to 300 and the proportion of the interim events was larger than 50%, the overestimation was acceptable. Moreover, even if the total number of events was 150, the overestimation was sufficiently small when the proportion of the interim events was >70% and a Pocock type α-spending function was used. The overestimation decreased when the total number of events and the proportion of the number of events in the interim analysis increased. In addition, the overestimation of the Pocock type α-spending function was smaller in comparison with that of the O'Brien-Fleming type, which is widely accepted for confirmatory trials.

We recommend setting the proportion of events in the interim analysis at 50% when the O'Brien-Fleming type α-spending function is used in confirmatory trials to reduce the risk of overestimation. In contrast, the Pocock type boundary could be used in explanatory trials.