Uncertainty-aware deep learning in healthcare: A scoping review

Area of Science:

• Artificial Intelligence in Medicine
• Machine Learning for Healthcare
• Trustworthy AI

Background:

• Deep learning (DL) implementation in healthcare faces mistrust due to lack of model certainty quantification.
• Entrustment requires conveying the probability of DL model output accuracy, an area with limited exploration.
• No consensus exists on optimal uncertainty quantification methods for DL in healthcare.

Purpose of the Study:

• Critically evaluate methods for quantifying uncertainty in DL for healthcare.
• Propose a conceptual framework for specifying DL prediction certainty.
• Enhance clinician and patient trust in healthcare AI applications.

Main Methods:

• Systematic literature search of Embase, MEDLINE, and PubMed databases.
• Adherence to PRISMA guidelines and validated tools for study quality rating.
• Data extraction using modified CHARMS criteria.

Main Results:

• 30 studies included; 24 focused on medical imaging using convolutional neural networks.
• Monte Carlo dropout was the predominant uncertainty quantification method in imaging.
• Conformal prediction demonstrated strong performance, ease of interpretation, and broad applicability.
• Non-imaging applications showed heterogeneous methods but effective uncertainty estimation for model comparison.
• Use of learning curves for epistemic uncertainty quantification was sparse.

Conclusions:

• Uncertainty estimation methods can identify critical DL misclassifications and compare models, fostering trust.
• Standardized guidelines for reporting performance and uncertainty metrics are crucial for field maturation.
• Conformal prediction offers a promising, interpretable approach for quantifying DL uncertainty across applications.