Applying the generalized partitioning principle to control the generalized familywise error rate

In multiple testing, strong control of the familywise error rate (FWER) may be unnecessarily stringent in some situations such as bioinformatic studies. An alternative approach, discussed by Hommel and Hoffmann (1988) and Lehmann and Romano (2005), is to control the generalized familywise error rate (gFWER), the probability of incorrectly rejecting more than m hypotheses. This article presents the generalized Partitioning Principle as a systematic technique of constructing gFWER-controlling tests that can take the joint distribution of test statistics into account. The paper is structured as follows. We first review classical partitioning principle, indicating its conditioning nature. Then the generalized partitioning principle is introduced, with a set of sufficient conditions that allows it to be executed as a computationally more feasible step-down test. Finally, we show the importance of having some knowledge of the distribution of the observations in multiple testing. In particular, we show that step-down permutation tests require an assumption on the joint distribution of the observations in order to control the familywise error rate.