Simultaneous confidence regions for closed tests, including Holm-, Hochberg-, and Hommel-related procedures

The derivation of simultaneous confidence regions for some multiple-testing procedures (MTPs) of practical interest has remained an unsolved problem. This is the case, for example, for Hochberg's step-up MTP and Hommel's more powerful MTP that is neither a step-up nor a step-down procedure. It is shown in this article how the direct approach used previously by the author to construct confidence regions for certain closed-testing procedures (CTPs) can be extended to a rather general setup. The general results are then applied to a situation with one-sided inferences and CTPs belonging to a class studied by Wei Liu. This class consists of CTPs based on ordered marginal p-values. It includes Holm's, Hochberg's, and Hommel's MTPs. A property of the confidence regions derived for these three MTPs is that no confidence assertions sharper than rejection assertions can be made unless all null hypotheses are rejected. Briefly, this is related to the fact that these MTPs are quite powerful. The class of CTPs considered includes, however, also MTPs related to Holm's, Hochberg's, and Hommel's MTPs that are less powerful but are such that confidence assertions sharper than rejection assertions are possible even if not all null hypotheses are rejected. One may thus choose and prespecify such an MTP, though this is at the cost of less rejection power.