An explainable and leakage-aware machine learning model for early risk stratification of radioiodine-refractory...

Radioiodine-refractory differentiated thyroid cancer (RAIR-DTC) poses a major therapeutic challenge, with loss of radioiodine avidity, limited treatment options, and worse outcomes. Current risk stratification systems were not designed to predict RAIR transformation before clinical refractoriness.

This retrospective two-center study included a development cohort of 302 consecutive patients and an external validation cohort of 122. Candidate predictors were restricted to baseline demographic, clinicopathological, molecular, and biochemical variables available at or before first postoperative RAI administration; all post-treatment, response-related, and time-dependent variables were excluded a priori. Six machine-learning algorithms were developed with repeated cross-validation and compared by discrimination, probability accuracy, and clinical utility. Model interpretation used SHapley Additive exPlanations (SHAP). External transportability was assessed by discrimination, calibration, decision-curve analysis, and validation of prespecified risk groups.

Forty-six patients (15.2%) developed RAIR-DTC in the development cohort and 18 (14.8%) in the external cohort. Random forest was selected for the most favorable balance of cross-validated discrimination and probability accuracy, despite comparable external discrimination from XGBoost and the stacked ensemble. It achieved an area under the receiver operating characteristic curve (AUC) of 0.896 and Brier score of 0.078 in the internal test cohort, with robust external discrimination (AUC, 0.892; PR-AUC, 0.597; sensitivity, 0.778; specificity, 0.798). Raw external calibration was poor (calibration slope 0.304; observed-to-expected ratio 0.613), indicating that ranking was preserved whereas absolute risk estimates required local recalibration. SHAP analysis identified age ≥55 years, TERT promoter (TERTp) mutation, vascular invasion, initial stimulated thyroglobulin (sTg), and extrathyroidal extension (ETE) as the dominant contributors. Training-derived quantile cut points generated three-tier risk groups, identifying a low-risk subgroup with zero events in the external cohort and a low event rate (3.3%) in the internal test set, while enriching the high-risk subgroup to event rates of 75.0% and 37.2%, respectively, with 95% confidence intervals of 47.6%-92.7% and 23.0%-53.3%, respectively.

An explainable, leakage-aware random forest based exclusively on routinely available baseline variables provided robust external discrimination and clinically meaningful early risk stratification for RAIR-DTC. Preserved discrimination did not guarantee transportable probability estimates, underscoring that external calibration and site-specific recalibration are essential prerequisites for responsible clinical deployment.