Confidence intervals and p-values for Williams' and other step-down multiple comparison tests against control

Area of Science:

• Biostatistics
• Statistical Methods
• Hypothesis Testing

Background:

• Traditional multiple comparison tests like Dunnett's allow p-values and confidence intervals.
• Step-down tests (e.g., Williams' test) traditionally report only statistical significance, limiting interpretation.
• A need exists for enhanced interpretability in step-down multiple comparison procedures.

Purpose of the Study:

• To propose approximate simultaneous confidence intervals for step-down multiple comparison tests.
• To assess the coverage properties of these proposed confidence intervals through simulation.
• To provide practicing statisticians with tools for improved interpretation of step-down test results.

Main Methods:

• Development of approximate confidence intervals for Williams' test, Dunnett's step-down test, and the closed t test.
• Simulation studies to evaluate the coverage performance of the proposed intervals.
• Calculation of p-values for the tested step-down procedures.

Main Results:

• The proposed simultaneous confidence intervals demonstrated good coverage properties.
• Coverage rates were typically between 94% and 96%, close to the nominal 95% level.
• P-values are readily calculable for these step-down tests with the proposed methods.

Conclusions:

• Approximate confidence intervals can be effectively implemented for step-down multiple comparison tests.
• These intervals provide valuable supplementary information for interpreting statistical significance.
• Practicing statisticians can now utilize both p-values and confidence intervals for enhanced analysis and reporting.