Analytical Uncertainty Propagation in Neural Networks

Area of Science:

• Computer Science
• Engineering
• Physics

Background:

• Machine learning, particularly neural networks, is widely applied across diverse domains.
• Estimating prediction certainty is vital for high-precision decision-making, yet uncertainty propagation in neural networks remains poorly understood.
• Measurement uncertainty, often expressed as error bars, is essential for applications demanding accuracy.

Purpose of the Study:

• To develop an analytical method for forward propagation of aleatoric uncertainty in neural networks.
• To provide a computationally inexpensive technique that leverages existing neural network weights and biases.
• To ensure provably correct uncertainty estimations without requiring modified training procedures.

Main Methods:

• The study adapts established analytical uncertainty propagation principles from physics and engineering.
• The method is designed to be applied post-training, utilizing only the neural network's weights and biases.
• No specialized training regimes are necessary for implementing this uncertainty quantification approach.

Main Results:

• The developed analytical method offers a way to quantify aleatoric uncertainty in neural network predictions.
• The approach was successfully applied to real-world semiconductor industry problems, including regression and image classification tasks.
• Demonstrated the generation of meaningful error bars, enhancing the reliability of machine learning in decision-making contexts.

Conclusions:

• The analytical method provides a robust and efficient solution for uncertainty propagation in neural networks.
• This technique enhances the trustworthiness of machine learning models by quantifying prediction certainty.
• The findings are particularly relevant for high-stakes applications in industries like semiconductor manufacturing where precise decision-making is critical.