Microbial Diversity Estimation and Hill Number Calculation Using the Hierarchical Pitman-Yor Process

Area of Science:

• Microbiology
• Computational Biology
• Statistical Ecology

Background:

• The human microbiome plays a crucial role in health, with species diversity as a key metric.
• Accurate diversity estimation is challenged by unobserved species in finite microbiome samples.
• Existing diversity measures can be biased by incomplete species detection.

Purpose of the Study:

• To develop a robust statistical model for human microbiome species abundance distributions.
• To derive accurate species diversity estimates, accounting for missing data.
• To provide a general framework for diversity quantification using Hill numbers within the HPY model.

Main Methods:

• Utilized the hierarchical Pitman-Yor (HPY) process to model species abundance across multiple populations.
• Employed a Gibbs sampler for efficient parameter estimation of the HPY model.
• Derived conditional and unconditional diversity estimates as functions of HPY parameters, including Hill numbers.

Main Results:

• The Gibbs sampler demonstrated effective performance in simulations for the HPY model.
• Conditional diversity estimates from the HPY model showed improvement over naive estimates when species were missing.
• HPY estimates outperformed naive estimates, particularly with smaller sample sizes, as shown in an infant gut microbiome dataset.

Conclusions:

• The HPY model provides a powerful tool for accurate microbiome diversity estimation, especially in the presence of unobserved species.
• Conditional diversity estimates offer a more reliable assessment of microbial community structure than traditional methods.
• The developed framework enhances the analysis of microbiome data, with implications for understanding host-microbe interactions and health.