Predicting lack of clinical improvement following varicose vein ablation using machine learning

Varicose vein ablation is generally indicated in patients with active/healed venous ulcers. However, patient selection for intervention in individuals without venous ulcers is less clear. Tools that predict lack of clinical improvement (LCI) after vein ablation may help guide clinical decision-making but remain limited. We developed machine learning (ML) algorithms that predict 1-year LCI after varicose vein ablation.

The Vascular Quality Initiative database was used to identify patients who underwent endovenous or surgical varicose vein treatment for Clinical-Etiological-Anatomical-Pathophysiological (CEAP) C2 to C4 disease between 2014 and 2024. We identified 226 predictive features (111 preoperative [demographic/clinical], 100 intraoperative [procedural], and 15 postoperative [immediate postoperative course/complications]). The primary outcome was 1-year LCI, defined as a preoperative Venous Clinical Severity Score (VCSS) minus postoperative VCSS of ≤0, indicating no clinical improvement after vein ablation. The data were divided into training (70%) and test (30%) sets. Six ML models were trained using preoperative features with 10-fold cross-validation (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). The algorithm with the best performance was further trained using intraoperative and postoperative features. The focus was on preoperative features, whereas intraoperative and postoperative features were of secondary importance, because preoperative predictions offer the most potential to mitigate risk, such as deciding whether to proceed with intervention. Model calibration was assessed using calibration plots, and the accuracy of probabilistic predictions was evaluated with Brier scores. Performance was evaluated across subgroups based on age, sex, race, ethnicity, rurality, median Area Deprivation Index, prior ipsilateral varicose vein ablation, location of primary vein treated, and treatment type.

Overall, 33,924 patients underwent varicose vein treatment (30,602 endovenous [90.2%] and 3322 surgical [9.8%]) during the study period and 5619 (16.6%) experienced 1-year LCI. Patients who developed the primary outcome were older, more likely to be socioeconomically disadvantaged, and less likely to use compression therapy routinely. They also had less severe disease as characterized by lower preoperative VCSS, Varicose Vein Symptom Questionnaire scores, and CEAP classifications. The best preoperative prediction model was XGBoost, achieving an AUROC of 0.94 (95% confidence interval [CI], 0.93-0.95). In comparison, logistic regression had an AUROC of 0.71 (95% CI, 0.70-0.73). The XGBoost model had marginally improved performance at the intraoperative and postoperative stages, both achieving an AUROC of 0.97 (95% CI, 0.96-0.98). Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.12 (preoperative), 0.11 (intraoperative), and 0.10 (postoperative). Of the top 10 predictors, 7 were preoperative features including VCSS, Varicose Vein Symptom Questionnaire score, CEAP classification, prior varicose vein ablation, thrombus in the greater saphenous vein, and reflux in the deep veins. Model performance remained robust across all subgroups.

We developed ML models that can accurately predict outcomes after endovenous and surgical varicose vein treatment for CEAP C2 to C4 disease, performing better than logistic regression. These algorithms have potential for important utility in guiding patient counseling and perioperative risk mitigation strategies to prevent LCI after varicose vein ablation.