Exploring the information in p-values for the analysis and planning of multiple-test experiments

Area of Science:

• Statistical methodology
• Hypothesis testing
• Bioinformatics

Background:

• Estimating the proportion of true null hypotheses is crucial for interpreting large-scale multiple testing.
• Existing methods may suffer from bias in estimating the null proportion and the density of effects under the alternative.

Purpose of the Study:

• To develop a new methodology for estimating the proportion of true null hypotheses and the density of effects under the alternative hypothesis.
• To introduce a generalized metric, the falsely interesting discovery rate (FIDR), for multiple hypothesis testing.
• To provide a robust framework for sample size calculations using the expected discovery rate (EDR).

Main Methods:

• Combines parametric modeling of p-value CDF with nonparametric spline modeling of the alternative hypothesis density.
• Employs penalized least squares with quadratic programming for efficient estimation.
• Proposes an estimator for the proportion of true nulls with reduced bias compared to marginal density estimators.

Main Results:

• The new methodology accurately estimates the density of effects under the alternative hypothesis.
• The proposed estimator for the proportion of true nulls demonstrates less bias than existing methods.
• The falsely interesting discovery rate (FIDR) is introduced as a generalization of the false discovery rate.

Conclusions:

• The developed methodology offers a more accurate and less biased approach to estimating null hypothesis proportions and alternative effect densities.
• The FIDR provides a valuable tool for interpreting results in large-scale hypothesis testing.
• The approach has practical applications in fields like gene expression analysis.