Mean comparisons and power calculations to ensure reproducibility in preclinical drug discovery

In the pharmaceutical industry, in vivo animal experiments are conducted to test the effects of novel preclinical drug compounds. Well-planned animal studies involve a sample size and statistical power analysis to provide a basis for the number of animals allocated into comparator arms of a future study. These calculations require approximate values for the parameters of a statistical model that will be applied to the future data and used to test for differences via statistical hypotheses. If the prestudy parameter estimates are nearly correct, the power analysis guarantees that a difference will be detected from the study data, up to a prespecified probability. Traditional power computations, however, are not calculated with reproducibility in mind. In this work, the issue of reproducibility in drug discovery is tackled from the point of view that study-to-study variability is not included in a typical sample size and power analysis. Three proposed methods that yield a reproducible mean-comparison analysis are derived and compared.