Testing pairwise contrasts in one-way analysis of variance designs

Area of Science:

• Behavioral Sciences
• Health Sciences
• Biostatistics

Background:

• One-factor analysis of variance (ANOVA) models are widely used in behavioral and health sciences.
• Researchers often need to compare means across independent groups or within a single group over time/conditions.
• While omnibus ANOVA tests are familiar, selecting appropriate pairwise multiple comparison procedures (MCPs) is less understood.

Purpose of the Study:

• To provide guidance on selecting pairwise multiple comparison procedures (MCPs) for one-factor ANOVA models.
• To discuss the critical issues of Type I error and statistical power in the context of multiple hypothesis testing.
• To aid researchers in making informed decisions for post-hoc mean comparisons.

Main Methods:

• The paper reviews the principles of hypothesis testing in ANOVA.
• It discusses the impact of Type I error rates and statistical power on MCP selection.
• Focuses on pairwise comparisons for independent samples, with a brief consideration of repeated measures.

Main Results:

• Selection of appropriate MCPs is crucial for accurate pairwise mean comparisons following a significant ANOVA.
• Understanding the trade-offs between controlling family-wise error rate and maintaining statistical power is essential.
• Different MCPs offer varying levels of protection against Type I errors and impact power.

Conclusions:

• Informed selection of MCPs enhances the validity of findings from one-factor ANOVA.
• Researchers should carefully consider Type I error control and power when choosing MCPs for pairwise comparisons.
• This guidance is applicable to both independent-sample and, with adjustments, repeated-measures ANOVA designs.