Comparative Performance of Clinician and Computational Approaches in Forecasting Adverse Outcomes in Intermittent...

Area of Science:

• Vascular Medicine
• Biostatistics
• Artificial Intelligence in Healthcare

Background:

• Machine learning (ML) shows promise in forecasting adverse cardiovascular and limb events in patients with intermittent claudication.
• This study is the first to directly compare ML predictive performance against traditional logistic regression (LR) and clinician judgment.

Purpose of the Study:

• To compare the predictive performance of ML models against logistic regression (LR) and clinicians.
• To evaluate the ability of different models to predict chronic limb-threatening ischemia (CLTI) and major adverse cardiovascular or limb events.

Main Methods:

• Utilized an anonymized dataset of 99 patients with 27 baseline characteristics.
• Assessed predictive performance using area under the receiver operating characteristic curve (AUC), F1 score, and Brier score.
• Compared a Least Absolute Shrinkage and Selection Operator (LASSO) based ML model with LR and predictions from 8 clinicians.

Main Results:

• ML significantly outperformed LR and clinicians across all predicted outcomes (CLTI, major adverse cardiovascular events, major adverse limb events).
• ML achieved higher AUC values (0.885-0.963) compared to LR (0.74-0.808) and clinicians (0.611-0.677).
• ML demonstrated superior accuracy with lower Brier scores (0.03-0.07) and higher F1 scores (0.80-0.86) than LR (Brier: 0.10-0.22, F1: 0.61-0.72) and clinicians (Brier: >0.31, F1: 0.50-0.59).

Conclusions:

• ML-based prediction models significantly outperform traditional regression and clinician judgment in managing intermittent claudication.
• ML's superiority stems from its ability to capture complex, nonlinear associations between patient variables.
• These findings highlight the potential of ML to improve risk stratification and patient outcomes in vascular disease.