Statistical reproducibility for pairwise t-tests in pharmaceutical research.

Area of Science:

• Statistics
• Biostatistics
• Pharmaceutical Research

Background:

• Statistical reproducibility is vital for validating research findings, especially in drug development.
• The t-test is widely used but its reproducibility requires robust assessment.
• Nonparametric predictive inference (NPI) offers a framework for evaluating reproducibility.

Purpose of the Study:

• To investigate and quantify the statistical reproducibility of the t-test using NPI.
• To assess the relationship between t-test reproducibility and common statistical measures (Cohen's d, p-value).
• To compare the reproducibility of the t-test with the Wilcoxon Mann-Whitney test.

Main Methods:

• Formulating reproducibility as a predictive inference problem.
• Developing and applying an NPI algorithm to calculate t-test reproducibility.
• Conducting simulations under null and alternative hypotheses.
• Applying NPI reproducibility to a preclinical drug efficacy experiment with multiple pairwise comparisons.

Main Results:

• The NPI algorithm successfully calculated t-test reproducibility across simulations and a real-world scenario.
• Reproducibility was analyzed in relation to Cohen's d and p-values.
• Comparisons revealed differences in reproducibility between the t-test and Wilcoxon Mann-Whitney test.
• Reproducibility was examined for dose selection decisions in preclinical trials.

Conclusions:

• NPI provides a robust framework for assessing statistical reproducibility of the t-test.
• Understanding reproducibility is critical for reliable decision-making in pharmaceutical research.
• This work advances methods for ensuring the reliability of statistical tests in scientific applications.