Failure Detection in Deep Neural Networks for Medical Imaging.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Machine Learning

Background:

• Deep neural networks (DNNs) are increasingly used in healthcare for diagnosis and prediction.
• The trustworthiness of DNNs is crucial, yet their confidence measures (softmax outputs) are often uncalibrated and unreliable, especially with noisy data.
• Detecting performance degradation and failure in deployed DNNs is essential for safe clinical integration.

Purpose of the Study:

• To develop and evaluate a method for detecting performance degradation and failure in DNNs used in medical settings.
• To leverage Bayesian deep neural networks (BDNNs) for calibrated confidence estimation and uncertainty quantification.
• To improve the reliability and trustworthiness of AI models in healthcare applications.

Main Methods:

• Employed Bayesian deep neural networks (BDNNs) to learn model parameter uncertainty, providing simultaneous predictions and confidence measures.
• Developed two failure detection methods: one using a fixed confidence threshold based on signal-to-noise ratio (SNR) and another using a neural network to learn the threshold.
• Tested the proposed methods on medical imaging datasets (PathMNIST, DermaMNIST, OrganAMNIST) under various noise conditions.

Main Results:

• BDNNs provide well-calibrated predictive confidence, outperforming standard DNNs under noisy conditions.
• The proposed failure detection methods effectively identify performance degradation by monitoring predictive confidence and variance.
• Accuracy improved by over 10% on unseen test samples due to the model abstaining from uncertain predictions, leading to fewer errors.

Conclusions:

• BDNNs coupled with proposed failure detection mechanisms enhance the trustworthiness and reliability of AI in medical imaging.
• Abstaining from low-confidence predictions improves overall model accuracy and safety in clinical settings.
• Monitoring predictive variance and confidence serves as an indicator of potential model failure or performance degradation.