Encoding the Latent Posterior of Bayesian Neural Networks for Uncertainty Quantification

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Bayesian Neural Networks (BNNs) are ideal for improving robustness and predictive uncertainty but are often unscalable or require restrictive assumptions.
• Existing BNNs are limited to small networks or rely on parameter independence, hindering their application in complex architectures.
• Deep Ensembles offer an alternative but incur significant computational costs due to linear scaling with the number of networks.

Purpose of the Study:

• To develop efficient deep Bayesian Neural Networks (BNNs) suitable for complex computer vision architectures and tasks.
• To reduce the assumptions on network parameters while maintaining scalability.
• To enable accurate uncertainty estimation and improve robustness in deep learning models.

Main Methods:

• Leveraged Variational Autoencoders (VAEs) to learn parameter interactions and latent distributions within each network layer.
• Introduced the Latent-Posterior BNN (LP-BNN) approach, compatible with the BatchEnsemble method for enhanced efficiency.
• Focused on enabling BNNs for large-scale computer vision models like ResNet-50 DeepLabv3+.

Main Results:

• LP-BNNs demonstrated high computational and memory efficiency during both training and testing.
• Achieved competitive performance across multiple metrics in challenging benchmarks for image classification and semantic segmentation.
• Showcased effectiveness in out-of-distribution detection tasks, highlighting improved model robustness.

Conclusions:

• LP-BNNs provide a scalable and efficient solution for incorporating Bayesian principles into deep neural networks.
• The VAE-based approach effectively models parameter distributions and interactions, overcoming limitations of traditional BNNs.
• This method offers a promising direction for robust and uncertainty-aware deep learning in computer vision.