Comparative Performance of Clinician and Computational Approaches in Forecasting Adverse Outcomes in Intermittent...

Recent evidence has shown that machine learning (ML) techniques can accurately forecast adverse cardiovascular and limb events in patients with intermittent claudication. This is the first study to compare the predictive performance of ML versus traditional logistic regression (LR) and clinicians.

An anonymized dataset of 99 patients with 27 baseline characteristics, compliance with best medical therapy/smoking cessation was used for comparison. Predictive performance was assessed using area under the receiver operating characteristic curve, F1 score, and Brier score. ML, LR, and clinicians were compared in their ability to predict outcomes including progression to chronic limb-threatening ischemia (CLTI) at 2 and 5 years, and probability of major adverse cardiovascular events or limb events upto 5 years. Independent variable importance ranking was performed to identify the most influential predictors.

The Least Absolute Shrinkage and Selection Operator based ML model was compared with (LR) and predictions from 8 clinicians. ML significantly outperformed LR and clinicians across all outcomes. Area under the receiver operating characteristic curve for CLTI at 2 years: ML 0.885, LR 0.74, best clinician 0.63; CLTI at 5 years: ML 0.936, LR 0.808, best clinician 0.639; major adverse cardiovascular event at 5 years: ML 0.963, LR 0.759, best clinician 0.611; major adverse limb event: ML 0.957, LR 0.9, best clinician 0.677. Brier scores for the ML model demonstrated excellent accuracy: ML (0.03-0.07), compared to LR (0.10-0.22) and clinicians (>0.31).The ML model demonstrated superior predictive performance with F1 scores ranging from 0.80 to 0.86 across all outcomes, consistently outperforming both LR (F1 scores: 0.61-0.72) and individual clinicians (F1 scores: 0.50-0.59).

ML-based prediction models significantly outperform traditional regression and clinician judgment, primarily due to their ability to capture complex nonlinear associations between variables.