Exact significance levels for multiple binomial testing with application to carcinogenicity screens

Area of Science:

• Biostatistics
• Experimental Design
• Toxicology

Background:

• Experimental designs often involve control and treatment groups.
• Statistical analysis frequently uses Fisher-Irwin exact tests for 2x2 tables.
• Multiple comparisons in these tests can inflate the risk of false positives.

Purpose of the Study:

• To develop a method for calculating the exact probability of at least one significant Fisher-Irwin test.
• To address concerns about overstated evidence due to multiple comparisons.
• To provide bounds for multiple-treatment group designs.

Main Methods:

• Exact permutational probability calculation for single treatment groups.
• Development of upper and lower bounds for multiple treatment groups.
• Application to carcinogen screening experiments.

Main Results:

• A method is provided to calculate the exact probability of significant findings.
• Bounds are established for designs with multiple treatment groups.
• The method is demonstrated with a carcinogen screening experiment example.

Conclusions:

• The proposed method offers accurate assessment of experimental evidence.
• It mitigates concerns regarding the overstatement of results from multiple Fisher-Irwin tests.
• This approach enhances the reliability of findings in carcinogen screening studies.