Interpretable Machine Learning Using Multimodal Ultrasound and MRI Features for Predicting Axillary Lymph Node...

The objective of the study is to develop and externally evaluate interpretable machine learning models integrating routinely reported ultrasound and MRI features for the preoperative prediction of axillary lymph node metastasis in patients with breast cancer. Clinical and imaging data from 673 patients with breast cancer examined between January 2019 and May 2025 were retrospectively analyzed. Data from the development center were divided at an approximately 70:30 ratio into a training cohort (n = 321) and an internal validation cohort (n = 137), while 215 patients from an independent institution constituted the external validation cohort. Feature selection was performed exclusively in the training cohort using Spearman rank correlation analysis. Ten supervised machine learning classifiers were fitted using the same three predictors and fixed parameter settings without additional hyperparameter optimization. Five-fold cross-validation was performed within the training cohort. Model performance was evaluated using the area under the receiver operating characteristic curve, sensitivity, specificity, positive-class F1 score, and decision curve analysis. SHapley Additive exPlanations were used for post-hoc interpretation of the Gaussian Naive Bayes model. In the internal validation cohort, logistic regression yielded the numerically highest AUC of 0.8766, followed by Gaussian Naive Bayes (0.8706) and decision tree (0.8640); the reported pairwise DeLong comparisons showed no statistically significant differences in AUC (all P > 0.05). In the external validation cohort, Gaussian Naive Bayes yielded the numerically highest AUC of 0.8154. Its AUC was significantly higher than those of LightGBM, AdaBoost, XGBoost, decision tree, K-nearest neighbors, and random forest (P < 0.05), but was not significantly different from those of logistic regression, multilayer perceptron, or support vector machine (P > 0.05). Gaussian Naive Bayes showed the highest estimated net benefit among the compared models over threshold probabilities of approximately 0.20-0.40, whereas logistic regression showed the highest estimated net benefit near 0.50. Among the three input predictors, SHAP analysis ranked lymph node cortical thickness as having the largest average contribution magnitude, followed by MRI-reported lymph node enlargement and tumor width. Gaussian Naive Bayes yielded the numerically highest AUC in the external validation cohort, although its discrimination was not significantly different from that of logistic regression, multilayer perceptron, or support vector machine. Its favorable external discrimination and higher estimated net benefit over part of the examined threshold range support further prospective evaluation as a preoperative ALNM risk-estimation approach.