Power of testing proportions in small two-sample studies when sample sizes are equal

Area of Science:

• Biostatistics
• Statistical Methods
• Experimental Design

Background:

• Investigators with small, equal-sized groups often encounter power calculation uncertainties due to reliance on asymptotic methods.
• Accurate power determination is crucial for effective experimental planning and valid statistical inference in small sample research.

Purpose of the Study:

• To present a method for determining statistical power for two-sided tests in small sample experiments.
• To compare the power and Type I error rates of Fisher's exact test (FET), the mid-P (MID) test, and the uncorrected chi-square test (CHI).

Main Methods:

• Comparison of three statistical tests: Fisher's exact test (FET), mid-P (MID), and uncorrected chi-square (CHI).
• Evaluation of power as a function of relative risk for each method.
• Assessment of relative power and Type I error rates across the tested methods.

Main Results:

• The uncorrected chi-square test (CHI) demonstrated the highest statistical power.
• The mid-P (MID) test exhibited intermediate power between CHI and Fisher's exact test (FET).
• Fisher's exact test (FET) was found to be the least powerful among the compared methods.
• Both CHI and MID occasionally exceeded the nominal alpha level, indicating potential for inflated Type I error rates in certain situations.

Conclusions:

• The choice of statistical test significantly impacts power in small sample experiments.
• The mid-P (MID) test offers a balance between power and Type I error control compared to FET and CHI.
• Researchers should be cautious of potential Type I error inflation when using CHI and MID with small sample sizes.