Testing each hypothesis marginally at alpha while still controlling FWER: how and when

Area of Science:

• Statistics
• Statistical Inference
• Hypothesis Testing

Background:

• Multiple hypothesis testing often inflates the familywise error rate (FWER).
• Rejecting hypotheses based solely on marginal p-values (≤ α) can lead to an unacceptable FWER.
• Controlling FWER is crucial for reliable scientific conclusions.

Purpose of the Study:

• To propose a new multiple testing procedure.
• To achieve strong familywise error rate control at level α.
• To enable individual hypothesis rejection at level α.

Main Methods:

• Incorporation of a prespecified consistency criterion into the testing algorithm.
• The criterion requires p-values to be below a threshold in one-sided testing.
• Extensions for two-sided testing and user-defined criteria.

Main Results:

• The proposed procedure successfully controls the familywise error rate at α in the strong sense.
• Individual hypotheses can be rejected at level α without inflating the overall error rate.
• The method is adaptable to different testing scenarios and user preferences.

Conclusions:

• The novel multiple testing procedure offers a robust solution for controlling FWER.
• It provides a reliable method for hypothesis rejection in complex statistical analyses.
• This approach enhances the validity of findings in multiple testing contexts.