How conservative is Fisher's exact test? A quantitative evaluation of the two-sample comparative binomial trial

Area of Science:

• Biostatistics
• Clinical Trial Design
• Statistical Methodology

Background:

• The optimal statistical method for two-sample comparative binomial trials remains debated.
• Fisher's exact test (FET) is favored by some practitioners for its conditional approach, fixing the total number of successes.
• Conditional methods can present interpretation challenges and lead to overly conservative results, increasing resource needs and patient exposure in clinical trials.

Purpose of the Study:

• To address the conservativeness of Fisher's exact test (FET) in two-sample comparative binomial trials.
• To develop a numerical algorithm for calculating the actual size (significance level) of FET.
• To propose adjusted significance levels (alpha(*)) to improve the efficiency of FET.

Main Methods:

• Development of a numerical algorithm to compute the size of FET for sample sizes up to 125 per group at alpha = 0.05.
• Utilizing the algorithm to define new significance levels, alpha(*), for each sample size n.
• Demonstration of statistical advantages through a sample size and power calculation example.

Main Results:

• The study provides a method to calculate the exact size of FET.
• New significance levels, alpha(*), are defined to bring the test size closer to the nominal alpha without exceeding it.
• The adjusted FET (using alpha(*)) shows statistical advantages in sample size and power calculations.

Conclusions:

• The numerical algorithm effectively calculates FET size and enables the definition of adjusted significance levels.
• Implementing the adjusted FET (alpha(*)) offers statistical advantages in two-sample comparative binomial trials.
• This approach can lead to more efficient trial designs with potentially fewer subjects exposed.