Uncertainty-Aware Machine Learning for Predicting Axillary Lymph Node Metastasis Using Breast MRI

Axillary lymph node metastasis (ALNM) is a significant prognostic factor in breast cancer and has an impact on staging, treatment and survival. The objective of this study is to create a machine learning model that will be able to predict axillary lymph node metastasis (ALNM) in a preoperative setting using breast MRI-derived and clinicopathological variables, while also achieving probability calibration and uncertainty-aware prediction for more reliable risk estimates.

For this retrospective single-centre study, 204 patients who underwent contrast-enhanced breast MRI from 2021 to 2024 were selected. The dataset comprised of 23 independent variables, respectively, representing demographic, clinical, radiological, histopathological, molecular characteristics along with a binary target variable indicating ALNM status. The data was split into 60% called training set, 20% calibration set, and 20% test set. Candidate models were evaluated based on ROC-AUC on the training subset that was used for screening. Subsequently, calibration was performed in a held-out calibration set, following which class-conditional conformal prediction was applied to quantify predictive uncertainty.

Out of all the models we evaluated, the Conformal-Calibrated Interpretable Risk Model (CCIRM) was found to be the best model, achieving a test accuracy of 0.9268, a weighted F1 score of 0.9270 and AUC of 0.937. With a precision of 0.9545 and a recall of 0.9130 in the ALNM-positive class, the model was potent. In addition to discrimination strength, the framework produces calibrated risk estimates and uncertainty-aware prediction sets that enable transparent interpretation of model outputs in clinically borderline cases.

The proposed approach includes the model selection based on the machine learning and probability calibration, as well as conformal prediction to achieve reliability, beyond label prediction, and uncertainty-aware risk estimation for preoperative ALNM assessment. The results from this analysis suggest that CCIRM is a potential methodological framework for trustworthy clinical decision support, although the prospective and multicentre external validation should be done before it becomes applicable to the clinical setting.