Measuring the Uncertainty of Predictions in Deep Neural Networks with Variational Inference.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Statistics

Background:

• Deep neural networks (DNNs) often lack uncertainty quantification, limiting their reliability in critical applications.
• Existing Bayesian deep learning methods can be computationally expensive and require optimizing many parameters.
• Accurate uncertainty estimation is crucial for robust decision-making and model diagnostics.

Purpose of the Study:

• To develop an efficient Bayesian approach for training deep neural networks.
• To enable reliable uncertainty quantification in model parameters with minimal overhead.
• To demonstrate the practical benefits of the proposed method in classification tasks.

Main Methods:

• A novel variational inference technique is employed to approximate the posterior distribution of network parameters.
• The approach represents posterior uncertainty per network layer, dependent on estimated parameter expectations.
• This results in optimizing only a few additional parameters compared to standard deep learning models.

Main Results:

• Achieved a 15% reduction in test error compared to classical deep learning on the MNIST dataset.
• Demonstrated the ability to compute credible intervals for predictions, aiding in uncertainty assessment.
• Obtained an average accuracy of 0.92 on a custom dataset using the GoogLeNet architecture, with 95% credible intervals detecting nearly all misclassifications.

Conclusions:

• The proposed Bayesian deep learning method offers an efficient way to quantify parameter uncertainty.
• The approach effectively reduces errors and enhances prediction reliability, applicable to large-scale networks.
• Uncertainty information can guide network architecture optimization and improve overall model performance.