A comment on futility monitoring

Futility monitoring of randomized clinical trials is becoming widely used. In this paper we discuss several concerns associated with aggressive monitoring for lack of activity in studies comparing experimental and control treatment arms: (1) stopping for futility when the experimental arm is doing better than the control arm, (2) conditional power not being low at the time of stopping, (3) potential loss of power to detect clinically interesting differences that are smaller than the design alternative, and (4) sensitivity of the power to the departure from the proportional hazards assumption. Our investigation suggests that aggressive futility rules do not generally reduce power (relative to less aggressive rules) under incorrect design assumptions such as overstatement of the target treatment effect or mild violations of the proportional hazards assumption. On the other hand, aggressive monitoring rules may result in an early termination for futility when the experimental arm is doing better than the control arm (in some cases nontrivially better, especially when trials are designed for unrealistically large effects). Thus, aggressive monitoring rules may fail to provide sufficiently convincing evidence to influence clinical practice or to establish a standard of treatment.