Reproducible Model Selection Using Bagged Posteriors

Area of Science:

• Statistics
• Computational Biology
• Machine Learning

Background:

• Bayesian model selection assumes data are generated from one of the proposed models.
• Model misspecification, where all models are incorrect, can lead to unstable Bayesian model selection and contradictory results.
• Existing methods lack robustness when dealing with misspecified models.

Purpose of the Study:

• To introduce and evaluate BayesBag, a novel approach to enhance the stability and reproducibility of Bayesian model selection.
• To address the challenges posed by model misspecification in Bayesian inference.
• To provide a more reliable method for model selection in practical applications.

Main Methods:

• Bagging on the posterior distribution (BayesBag) by averaging posterior model probabilities over bootstrapped datasets.
• Theoretical analysis of the asymptotic behavior of the bagged posterior under model misspecification.
• Empirical assessment using synthetic and real-world data for feature selection and phylogenetic tree reconstruction.

Main Results:

• BayesBag significantly improves reproducibility and reliably assigns posterior mass to optimal models when all models are misspecified.
• Compared to the standard Bayesian posterior, BayesBag is more conservative under correct model specification.
• The proposed method demonstrates enhanced stability and reproducibility over traditional Bayesian model selection.

Conclusions:

• BayesBag offers an easy-to-use and broadly applicable solution to improve Bayesian model selection.
• The method enhances stability and reproducibility, particularly in scenarios with model misspecification.
• BayesBag provides a valuable alternative to standard Bayesian model selection, yielding more trustworthy results.