A power study of a sequential method of p-value adjustment for correlated continuous endpoints

In the course of clinical (or preclinical) trial studies, it is a common practice to conduct a relatively large number of tests to extract the maximum level of information from the study. It has been known that as the number of tests (or endpoints) increases, the probability of falsely rejecting at least one hypothesis also increases. Single-step methods such as the Bonferroni, Sidák, or James approximation procedure have been used to adjust the p-values for each hypothesis. To reduce the conservatism (i.e., underestimating type I error) possessed by the aforementioned methods, Holm proposed a so-called "free-step-down" procedure. This adjustment can be made even less conservative by incorporating the dependence structure of endpoints at each adjustment step of the procedure. That is done by sequentially applying James's approximation procedure for correlated endpoints at each step, referred to as the Free-James method. This article primarily compares the power of the Free-James method to the power of the Bonferroni and James single-step-down and the Holm free-step-down methods. Two definitions of power are considered: (a) the probability of correctly rejecting at least one hypothesis when it is true, and (b) the probability of correctly rejecting all hypotheses that are true. Monte Carlo simulations show that the Free-James method is as good as other methods under definition (a) and the most powerful under definition (b) for various sample sizes, numbers of endpoints, and correlations.