Demonstration of the reproducibility of treatment efficacy from a single multicenter trial

According to the Food and Drug Administration's Guidelines for the Format and Content of the Clinical and Statistical Sections of New Drug Applications, approval of a new drug "should be supported by more than one well-controlled trial and carried out by independent investigators. This interpretation is consistent with the general scientific demand for replicability." Nevius has described a four-point proposal for assessing statistical evidence in a single multicenter trial. Briefly, these four points are: (1) combined analysis shows significant results, (2) consistency over centers in terms of direction, (3) consistency over centers in terms of producing nominally significant results in centers with sufficient power, and (4) evidence of efficacy after adjustment for multiple comparisons. What is not clear from Nevius' proposal is how to quantify whether the amount of evidence in a single multicenter trial is equivalent to that from two separate trials. It is proposed that the post hoc subdivision of a multicenter trial may address this issue if the inherent multiple testing problem is accommodated. A minimax statistic is developed to test the hypothesis that the effect of the drug has been reproduced in a single multicenter trial. Monte Carlo simulation is used to generate the distribution of the minimax statistic under the null and several alternative hypotheses. Data from a multicenter trial are used to demonstrate the technique. Bootstrapping is used to determine the null distribution of the minimax statistic.