Uncertainty Modeling Outperforms Machine Learning for Microbiome Data Analysis

Area of Science:

• Microbiology
• Bioinformatics
• Computational Biology

Background:

• Microbiome sequencing quantifies relative, not absolute, microbial abundances.
• Existing normalization methods rely on assumptions that can introduce bias.
• Direct microbial load measurements are accurate but costly and infrequent.

Purpose of the Study:

• To evaluate the efficacy of machine learning in predicting microbial load from sequencing data alone.
• To assess the generalizability of machine learning models across diverse microbiome studies.
• To compare machine learning approaches with alternative methods for handling microbial load uncertainty.

Main Methods:

• Assembled 'mutt,' the largest database of paired sequencing and microbial load measurements (35 studies, >15,000 samples).
• Evaluated published machine learning models on the 'mutt' database and benchmark datasets.
• Implemented and compared Bayesian partially identified models for propagating scale uncertainty.

Main Results:

• Machine learning models demonstrated poor generalization, performing worse than a naive baseline on average.
• Model failures were attributed to covariate shift, limited shared taxa, compositional differences, and preprocessing variations.
• Bayesian partially identified models consistently outperformed normalization and machine learning methods across 30 benchmark datasets.

Conclusions:

• Machine learning approaches are unreliable for predicting microbial load from microbiome sequencing data.
• Bayesian partially identified models provide a principled and reproducible method for accounting for scale uncertainty in microbiome inference.
• The 'mutt' database serves as a valuable resource for evaluating microbiome analysis methods.