Evaluating predictive performance and generalizability of traditional and artificial intelligence models in...

Surgical site infections (SSIs) are a significant complication following spinal surgery. These infections contribute to increased morbidity, prolonged hospital stays, and substantial healthcare costs. Traditional statistical models have been widely used to predict SSI risk, but artificial intelligence (AI) and its machine learning (ML) methods have also been used for SSI prediction.

This systematic review aims to evaluate the predictive accuracy of AI models versus traditional statistical models in assessing SSI risk following spinal surgery.

A systematic review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

We searched Medline, Embase, Scopus, Web of Science, and ClinicalTrials.gov. Studies were included if they developed predictive models for SSI following spinal surgery using either AI or traditional statistical approaches. Risk of Bias for all studies was assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST). Predictive model performance was compared using metrics such as the C-statistic and Area under the Receiver Operating Characteristic curve (AUC-ROC).

A total of 51 studies were included. Among these, 42 studies used traditional statistical methods, while 9 used AI / ML models. Logistic regression was the most common method among traditional models (95.2%). Across the ML studies, all of which used supervised models trained on tabular data, decision‑tree-based and linear algorithms (n=7, 77.8% each) were the most common, followed by neural networks and support vector machines (n=4, 44.4% each). Traditional models achieved a C-statistic between 0.7 and 0.8 in 40.5% of cases (n=17), with only 4.8% (n=2) exceeding 0.9. AI models showed a C-statistic of 0.9 or higher in 44.4% of cases (n=4). However, 77.8% of those ML-based models (n=7) performed an internal cross validation and only 33.3% reported calibration data (n=3), and none of the studies are externally validated, which raises important concerns about their current clinical applicability and generalizability.

This systematic review, the first of its kind, observed that studies utilizing the AI models reported a potential for excellent classification accuracy in predicting SSI following spinal surgery. However, the current shortcomings in methodology limit their generalizability and immediate clinical implementation. For existing models, most AI studies remain in the early stages of development and its findings in excellent performance should be taken with caution. This review highlights the need for standardized model benchmarking and employing external validation to reliably assess generalizability. Furthermore, advancing beyond conventional tabular data by incorporating state-of-the art AI models that leverage multimodal data could significantly expand the potential of predictive analytics in this domain - thus help guide clinical decision making.