Mixture-based gatekeeping procedures for multiplicity problems with multiple sequences of hypotheses

Complex multiplicity problems arise in drug development programs with several sets of clinical objectives. This article considers a common setting with two sources of multiplicity induced by the analysis of multiple dose levels based on ordered endpoints. This results in multiplicity problems with multiple sequences of null hypotheses of no effect. Type I error rate inflation in problems of this type is typically addressed by using gatekeeping procedures that account for the hierarchical structure of the trial objectives. A general method for building gatekeeping procedures, known as the mixture method, tends to be conservative in problems with several sequences of hypotheses. This article defines a modified mixture method and shows that this method provides a power advantage over the standard mixture method. In addition, it is demonstrated that in special cases the modified mixture method allows for a stepwise testing algorithm, which facilities the implementation of gatekeeping procedures and general decision making. The new methodology is illustrated using two clinical trial examples.