Uncertainty propagation for dropout-based Bayesian neural networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Statistics

Background:

• Deep neural networks (DNNs) require robust uncertainty evaluation for real-world applications, especially with unexpected data.
• Bayesian inference and Bayesian neural networks (BNNs) are key for uncertainty assessment.
• Monte Carlo Dropout (MC Dropout) is a common but computationally intensive method for BNN uncertainty estimation.

Purpose of the Study:

• To develop a computationally efficient, sampling-free method for uncertainty evaluation in DNNs.
• To enable reliable uncertainty estimation for safety-critical systems.
• To extend uncertainty evaluation methods to various neural network architectures, including recurrent neural networks (RNNs).

Main Methods:

• Proposed a sampling-free technique to convert dropout-trained neural networks into Bayesian neural networks.
• Implemented variance propagation for uncertainty quantification.
• Validated the method on feed-forward DNNs and recurrent neural networks (RNNs) like LSTMs.

Main Results:

• Demonstrated significant computational efficiency compared to traditional MC Dropout.
• Achieved reliable statistical performance in uncertainty detection.
• Successfully applied the method to language modeling and out-of-distribution detection tasks.

Conclusions:

• The proposed sampling-free method offers an efficient and reliable alternative for uncertainty evaluation in DNNs.
• This technique is applicable to diverse neural network architectures, enhancing their practical utility.
• The method improves the safety and reliability of AI systems in critical applications.