Multiple comparisons: comparisonwise versus experimentwise Type I error rates and their relationship to power.

Area of Science:

• Statistics
• Biostatistics
• Experimental Design

Background:

• The experimentwise Type I error rate is often considered paramount in multiple comparison procedures.
• Type I errors and their implications in biological experiments warrant careful consideration.
• The trade-off between Type I error control and statistical power is a critical aspect of experimental design.

Purpose of the Study:

• To challenge the prevailing emphasis on experimentwise Type I error rates.
• To discuss the significance and occurrence of Type I errors in biological research.
• To evaluate the impact of Type I error protection on statistical power.

Main Methods:

• Discussion of the importance and occurrence of Type I errors.
• Explanation of comparisonwise versus experimentwise Type I error rates.
• Analysis of the effect of error rate control on statistical power.

Main Results:

• Contention that experimentwise Type I error rate is the most important attribute is challenged.
• Biological experiments may benefit from procedures controlling comparisonwise error rates.
• Procedures like Fisher's LSD and Duncan's test, despite higher experimentwise rates, offer greater power.

Conclusions:

• Controlling comparisonwise Type I error rates can enhance statistical power in detecting treatment differences.
• Researchers should consider the balance between Type I error control and power when selecting multiple comparison procedures.
• The choice of error rate control (comparisonwise vs. experimentwise) depends on the specific research goals and context.