P-value calibration for multiple testing problems in genomics

Conservative statistical tests are often used in complex multiple testing settings in which computing the type I error may be difficult. In such tests, the reported p-value for a hypothesis can understate the evidence against the null hypothesis and consequently statistical power may be lost. False Discovery Rate adjustments, used in multiple comparison settings, can worsen the unfavorable effect. We present a computationally efficient and test-agnostic calibration technique that can substantially reduce the conservativeness of such tests. As a consequence, a lower sample size might be sufficient to reject the null hypothesis for true alternatives, and experimental costs can be lowered. We apply the calibration technique to the results of DESeq, a popular method for detecting differentially expressed genes from RNA sequencing data. The increase in power may be particularly high in small sample size experiments, often used in preliminary experiments and funding applications.