Using controls to limit false discovery in the era of big data

Area of Science:

• Statistics
• Bioinformatics
• Genomics

Background:

• Traditional false discovery rate (FDR) control methods in high-dimensional statistics rely on accurate p-values and extrapolation of null distributions.
• These intermediate steps can be challenging and compromise the reliability of statistical results.
• The multiple comparisons problem is a significant concern in big data analysis.

Purpose of the Study:

• To develop a general FDR controlling method suitable for big data settings.
• To overcome limitations of current FDR procedures, specifically the reliance on p-values and distribution extrapolation.
• To leverage abundant control data in big data studies for more robust inference.

Main Methods:

• A novel FDR control method is presented that uses control data to empirically construct the null distribution.
• The empirical null distribution is directly compared to the empirical test data distribution.
• The method avoids the calculation of p-values and extrapolation into unknown distribution tails.

Main Results:

• The proposed control data-based empirical FDR procedure provides a more direct comparison between test and control data.
• This approach aligns with fundamental scientific principles of inference.
• The method is successfully demonstrated in a structural genomics application.

Conclusions:

• A general statistical framework for FDR control tailored to big data is established.
• The method bypasses potentially problematic modeling and extrapolation steps by using empirical distributions and control data.
• The procedure is broadly applicable where controlled experiments or internal negative controls are available, common in big data research.