Tackling external validation challenges: experience with normal tissue complication probability (NTCP) models for...

Aim of this study is to validate with an external independent cohort of patients previously published Normal Tissue Complication Probability (NTCP) models for radiation-related toxicity in head and neck cancer patients treated with curative radiotherapy (± associated systemic treatments).

Patients treated from 2010 to 2022 at IEO were retrospectively reviewed. Five NTCP models for the following endpoints were evaluated: I) physician-rated swallowing dysfunction grade (G) 2-4, 6 months after RT; II) tube-feeding dependence (TFD) 6 months after RT; III) acute oral mucositis (OM) G ≥ 3 at any time during RT; IV) OM during RT treatment, mean G ≥ 1.5; V) G ≥ 2 laryngeal edema within 15 months from RT. External validation of the selected models on our cohorts was assessed evaluating the discriminating ability (in terms of Area Under the Receiver Operating Characteristic curve (AUC), Brier score) and the calibration (in terms of calibration intercept, calibration slope, and finally through the Hosmer-Lemeshow goodness-of-fit test).

A total of 97 patients met the inclusion criteria for dysphagia (5% events); 88 patients for TFD (3% events); 113 patients for OM G ≥ 3 (42% events); 114 patients for OM mean G ≥ 1.5 (63% events) and 102 patients for G ≥ 2 laryngeal edema (21% events). For all considered endpoints and all the considered models, the calculated NTCP was higher for patients who reported the corresponding toxicities, despite the difference was statistically significant for laryngeal edema only. Best-performing models were TFD (AUC = 0.73, CI: 0.24 - 1.22) and physician-rated swallowing dysfunction (AUC = 0.68, CI: 0.52 - 0.85), even if in both cases the low number of events influences the precision of the estimate, resulting in wide confidence intervals.

Overall, results retrieved from our analysis confirm that NTCP model can be applied in populations with characteristics similar to those of the training cohorts. Nevertheless, our data also confirmed that external validation remains a challenge in particular when the numbers of events are low and patients' characteristics between the training and validating cohorts are different.