Basic ideas and concepts for multiple comparison procedures

Around 1970, the author proposed a general theoretical approach to multiple decision problems (MDPs) of which multiple comparison problems (MCPs) are special cases. Suppose that a sample space Χ is given together with a set of probability distributions P = {P(θ), θ ∈ Ω} defined over Χ. Let a finite partition of the parameter space Ω = cupω(a), a ∈ A be given. Based on the observation X ∈ X, an MDP is to decide, which ω(a) the true parameter θ belongs to. An MD confidence procedure is a mapping ψ from X to the class of subsets of A, such that the probability that cupω(a), ω(a) C ψ(X) includes the true parameter θ is not smaller than 1-α(θ) . Here, 1-α(θ) is called the level of the confidence procedures and may vary depending on θ∈ω(a) . The MP confidence procedures are derived from the following proposition. When the ω(a) 's are mutually disjoint, there is a one-to-one correspondence between an MD confidence procedure ψ and a collection of (non-randomized) tests for the hypotheses H(a) : θ∈ω(a) with level α(a) by rejecting the hypothesis H(a) if ω(a) ∉ ψ(X). In this paper we discuss in detail the problems of determining the signs or the orderings of normal means. The resulting confidence procedures from the LR tests are seen to be too complicated and difficult to understand. We therefore propose simplified, less powerful methods. We define an overlapping partition of Ω into simple sets, such that the original ω(a) 's can be expressed as an intersection of such simple sets. For each such set we define rejection regions corresponding to the levels α, α/2,...,α/k. Then we obtain the acceptance regions for H(a)  : θ∈ω(a) given as the intersection of all acceptance regions for the simple sets containing ω(a) at the level α/k, if there are k such simple sets. This method can be extended to obtain sequential confidence procedures.