A Review of Uncertainty Representation and Quantification in Neural Networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Uncertainty Quantification

Background:

• Neural networks require robust uncertainty estimation for dependable predictions.
• Distinguishing between aleatoric (data) and epistemic (model) uncertainty is key.
• Current methods for uncertainty quantification in deep learning are diverse.

Purpose of the Study:

• To provide a comprehensive overview of methodologies for uncertainty representation and quantification in neural networks.
• To differentiate between aleatoric and epistemic uncertainty within neural network models.
• To analyze the strengths and limitations of various uncertainty estimation techniques.

Main Methods:

• Review of classical probabilistic techniques: Bayesian neural networks and deep ensembles.
• Exploration of generalized probability methods: Dirichlet distributions, belief functions, random sets, probability intervals, and credal sets.
• Examination of interval-based approaches using interval models.

Main Results:

• Categorization of diverse uncertainty quantification methodologies.
• Analysis of the applicability and constraints of each approach.
• Identification of research gaps and future directions in the field.

Conclusions:

• Effective uncertainty estimation is vital for trustworthy AI systems.
• A variety of methods exist, each with unique advantages and disadvantages.
• Further research is needed to advance robust uncertainty quantification in neural networks.