Accuracy of Artificial Intelligence-Based Models Versus Conventional Scoring Systems (APACHE, SOFA, and SAPS) in...

Accurate prediction of mortality in critically ill patients admitted to the ICU is essential for clinical decision-making and resource allocation. Conventional scoring systems such as Acute Physiology and Chronic Health Evaluation (APACHE), Sequential Organ Failure Assessment (SOFA), and Simplified Acute Physiology Score (SAPS) are widely used but are limited by their static structure and linear assumptions. Artificial intelligence (AI)-based models offer more flexible, data-driven approaches; however, their comparative performance remains uncertain. This systematic review evaluated the performance of AI-based models compared with conventional ICU scoring systems for predicting in-hospital mortality. A systematic search of PubMed, the Excerpta Medica database (Embase), Web of Science, and Scopus was conducted from January 2015 to August 2025. Based on predefined eligibility criteria, studies comparing AI-based models with conventional scoring systems and reporting performance metrics such as area under the receiver operating characteristic curve (AUC), sensitivity, or specificity were included. Risk of bias was assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST; Cochrane Prognosis Methods Group and the PROBAST Steering Group, University of Bristol, Bristol, United Kingdom, and collaborating international experts under the Cochrane Collaboration). A descriptive synthesis of discriminative performance was performed, along with a quantitative diagnostic test accuracy synthesis using a bivariate random-effects (Reitsma) model. Exploratory subgroup analyses using inverse variance-weighted random-effects meta-analysis with the DerSimonian-Laird estimator were conducted based on model type, dataset characteristics, and temporal modeling. Ten studies involving approximately 500,000 ICU admissions were included. AI-based models demonstrated higher discriminative performance than conventional scoring systems, with reported AUC values ranging from 0.83 to 0.97 and ΔAUC ranging from 0.04 to 0.19. Eight studies contributed to the quantitative diagnostic test accuracy synthesis, yielding a pooled sensitivity of 0.845 (95% CI: 0.815-0.871) and a pooled specificity of 0.791 (95% CI: 0.728-0.843). Subgroup analyses demonstrated progressively higher pooled AUC values among tree-based/ensemble and deep learning models compared with classical machine learning (ML) approaches. Temporal and longitudinal models demonstrated pooled performance comparable to static variable-based models, while single-center cohorts demonstrated higher pooled AUC values than multicenter datasets. Calibration reporting was heterogeneous and not suitable for quantitative synthesis. Overall, AI-based models show improved discriminative performance for mortality prediction in critically ill patients; however, substantial heterogeneity in study design, validation methodology, and reporting standards highlights the need for further external validation before routine clinical implementation.