DropConnect is effective in modeling uncertainty of Bayesian deep networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep neural networks (DNNs) excel in critical domains like medical diagnosis and autonomous driving.
• Ensuring safety requires understanding DNN limitations and potential errors.
• Existing Bayesian deep network approaches face computational challenges with complex architectures.

Purpose of the Study:

• To develop a computationally tractable framework for approximating Bayesian inference in DNNs.
• To enable DNNs to estimate their own uncertainty, indicating potential errors.
• To enhance the safety and reliability of deep learning in sensitive applications.

Main Methods:

• Imposing a Bernoulli distribution on model weights to approximate Bayesian inference.
• Introducing Monte Carlo DropConnect (MC-DropConnect) for uncertainty estimation.
• Developing new metrics for quantifying and comparing uncertainty estimates.

Main Results:

• MC-DropConnect effectively represents model uncertainty with minimal structural or computational overhead.
• Extensive validation across diverse network architectures and datasets (classification, semantic segmentation).
• Significant improvements in both prediction accuracy and uncertainty estimation quality compared to state-of-the-art methods.

Conclusions:

• MC-DropConnect provides a practical solution for uncertainty quantification in DNNs.
• The method enhances the safety of deep learning applications in critical domains.
• MC-DropConnect offers superior performance in accuracy and uncertainty estimation over existing approaches.