Evaluation of Machine Learning Model Performance in Diabetic Foot Ulcer: Retrospective Cohort Study.

Area of Science:

• Medical Informatics
• Computational Biology
• Clinical Decision Support Systems

Background:

• Diabetic foot ulcers (DFUs) are a significant multifactorial problem with severe outcomes.
• Machine learning (ML) offers potential for recognizing complex disease patterns and aiding clinical decisions.
• DFUs serve as an ideal case for developing a comprehensive ML implementation framework.

Purpose of the Study:

• To establish a framework for the appropriate application of ML algorithms in predicting multifactorial disease outcomes.
• To evaluate the predictive performance of various ML models for diabetic foot ulcer healing.
• To assess the clinical utility and calibration of ML models in a real-world DFU dataset.

Main Methods:

• A DFU dataset was used to compare ML models, including logistic regression, SVM, k-nearest neighbors, RF, XGBoost, BART, and ANN.
• Patient characteristics were selected using statistical tests (ANOVA, chi-square) and expert validation.
• Data imputation (MIDAS Touch) and balancing (adaptive synthetic sampling) were performed prior to model training and cross-validation.

Main Results:

• The SVM model achieved the highest accuracy (0.853) and AUC (0.922) in predicting DFU healing.
• Random Forest (RF) and XGBoost also showed strong performance, with accuracies of 0.838 and 0.815, respectively.
• All top-performing models (SVM, RF, XGBoost) demonstrated good calibration and clinical utility via Brier scores and DCA.

Conclusions:

• Effective handling of missing values, feature selection, and class imbalance are crucial for clinical ML applications.
• The SVM model exhibited superior predictive power for DFU healing compared to other evaluated ML algorithms.
• The study highlights the potential of calibrated ML models, like SVM, for enhancing clinical decision-making in DFU management.