Uncertainty Quantification for Deep Learning-based Medical Imaging Classification Model Evaluation and Individualized...

Deep neural networks (DNNs) have been applied in numerous medical image analysis tasks, yet the lack of reliable uncertainty quantification (UQ) limits safe clinical deployment. In high-stakes medical decision making, model overconfidence can lead to severe consequences, underscoring the need for methods that evaluate and interpret model behaviors. In this work, we propose two methods that extend the prediction uncertainty to characterize deep learning model behaviors and assess sample-level prediction risk (i.e., error rate). Specifically, one method is for model-level evaluation that compares UQ calibration across medical imaging classification models using the Risk-Coverage Curve, and another method is for risk estimation at the sample level through translating raw uncertainty scores into context interpretable prediction error probabilities for individual samples. The two methods can utilize either the softmax probability from a standard convolutional neural network or the variance-based uncertainty outputs from a newly customized heteroscedastic binary cross-entropy (HBCE) model. We performed experiments on a breast cancer imaging dataset for a binary classification model for computer-aided diagnosis. Results show the feasibility and promising effects of our methods in evaluating the overall calibration of UQ and individualized risk. The HBCE-generated uncertainty provides superior efficacy compared to the standard softmax probability in our dataset and classification task. These results highlight the potentials of the proposed methods as a practical tool for clinical utility-supporting 1) systematic model evaluation for diagnostic model selection and benchmarking under varying clinical settings and 2) individualized uncertainty interpretation to assist model users (e.g., radiologists) for personalized decision making when working with models. This work contributes new methods and tools that can enhance human-AI collaboration for medical imaging tasks.