Exact significance levels for multiple binomial testing with application to carcinogenicity screens

A simple experimental design consisting of one control group and one or more treatment groups is considered. Relevant research often focuses on the presence or absence of any of several characteristics in the treatment group(s). The statistical analysis frequently includes the comparison of the control group with each treatment group by the use of Fisher-Irwin exact tests for each of many 2 x 2 tables. The multiplicity of comparisons has given rise to concern that individual Fisher-Irwin tests could seriously overstate the experimental evidence in some situations. This paper provides a method for calculating the exact permutational probability of at least one significant Fisher-Irwin test when only one treatment group and one control group is used. For multiple-treatment-group designs, upper and lower bounds on the probability are provided. Emphasis is given throughout to carcinogenesis screening experiments and an example of such an experiment is provided.