Development and evaluation of an explainable machine learning selection pipeline for predicting neoadjuvant
Yijun Xu1,2, Wei Gong1, Lei Ji1
1Department of Radiation Oncology, The First Affiliated Hospital of Soochow University, Suzhou, 21500, China.
Background:
The selection of appropriate machine learning (ML) methods for clinical research remains challenging, particularly when both predictive performance and model explainability are required in small-sample datasets. Conventional approaches often rely on limited variables and expert-driven choices, whereas many ML models remain difficult to justify clinically. This study aimed to develop an explainable ML model selection pipeline and demonstrate its application in predicting neoadjuvant chemoradiotherapy (nCRT) response in locally advanced rectal cancer (LARC).
Methods:
We proposed a six-stage explainable ML model selection pipeline comprising data selection and preprocessing, algorithm pool construction, model training, model evaluation, model explanation, and an internal clinical logic consistency check (non-deployable sanity check). The workflow was applied to a retrospective cohort of 128 patients with LARC treated with nCRT. Four ML algorithms, including support vector machine (SVM), decision tree (DT), random forest (RF), and logistic regression (LR), were evaluated. Model performance was assessed using accuracy, F1-score, AUROC, AUPRC, bootstrap confidence intervals, and five-fold cross-validation, followed by explainability screening using SHAP.
Results:
Under predefined screening criteria, DT models achieved the most favorable balance between predictive performance and explainability. Using pretreatment tumor markers alone, the selected DT models achieved an accuracy of 0.82 and F1-score of 0.71 for pathological complete response (pCR) prediction, and an accuracy of 0.76 and F1-score of 0.72 for tumor regression grade (TRG) prediction. SHAP analysis consistently identified carcinoembryonic antigen (CEA) and carbohydrate antigen 19-9 (CA19-9) as the most influential predictors, and lower baseline levels of these markers were associated with better pathological response.
Conclusions:
This study provides a practical and reproducible framework for selecting interpretable ML models in small clinical datasets. In the present LARC case study, DT-based models showed acceptable discrimination and transparent decision logic, while tumor markers emerged as clinically plausible predictors of nCRT response. The proposed workflow may support future multi-center validation and broader application in clinically interpretable predictive modeling.
