A broken promise: microbiome differential abundance methods do not control the false discovery rate

Area of Science:

• Microbiome research
• Statistical bioinformatics
• Human health and disease

Background:

• High-throughput sequencing enables human microbiome characterization.
• Statistical methods for microbiome data analysis are underdeveloped.
• Differential abundance methods are crucial for biomarker discovery and drug efficacy studies.

Purpose of the Study:

• To benchmark the performance of popular statistical methods for microbiome data analysis.
• To identify potential issues with current statistical approaches.
• To introduce a novel simulation tool for improved method evaluation.

Main Methods:

• Extensive parametric and nonparametric simulations were performed.
• Real data shuffling algorithms were employed for validation.
• A new simulation tool was developed to generate correlated count data.

Main Results:

• A significant excess of false discoveries was observed across multiple methods.
• Results were consistent across simulation and real data analyses.
• Species correlation negatively impacts statistical method performance.

Conclusions:

• Current statistical methods for microbiome analysis may be unreliable.
• Concerns exist regarding the reproducibility of microbiome studies.
• Further development and rigorous benchmarking of statistical tools are essential.