Two-sample t -test for testing hypotheses in small-sample experiments

Area of Science:

• Molecular Biology
• Biostatistics
• Medical Research

Background:

• Approximately 50% of biological discoveries are irreproducible, partly due to low statistical power in small sample sizes common in molecular biology and medicine.
• Traditional two-sample t-test has limited power with small samples, hindering the reliability of findings.
• Enhancing statistical power is not always feasible for small-sample experiments; reducing type I error rates offers an alternative.

Purpose of the Study:

• To introduce and evaluate a novel statistical test, the t-test, designed to reduce type I error rates in small-sample experiments.
• To compare the performance of the t-test against the traditional t-test and Wilcoxon test for improving reproducibility.

Main Methods:

• Theoretical analysis and large-scale simulation studies were conducted to assess the t-test's performance.
• The t-test was compared with the t-test and Wilcoxon test using both simulated data and real experimental datasets, including microarray data.
• Mathematical derivations of the density distribution and probability cumulative function of the t-statistic were performed.

Main Results:

• The t-test significantly reduced type I error rates compared to the t-test and Wilcoxon test in small-sample experiments.
• The t-test demonstrated empirical power comparable to the t-test.
• Analysis of p-value density distribution explained the lower type I error rate of the t-test.
• Real experimental data and a microarray dataset confirmed the superior performance of the t-test over the t-test and other methods.

Conclusions:

• The t-test is a more effective statistical method than the t-test and Wilcoxon test for enhancing the reproducibility of small-sample experiments in molecular biology and medicine.
• The t-test offers a viable strategy to mitigate irreproducibility by controlling type I error rates.
• The theoretical and observed distributions of the t-statistic were found to be well-matched, validating the test's mathematical underpinnings.