External validation of an artificial intelligence model using clinical variables, including ICD-10 codes, for...

Artificial intelligence (AI) is being increasingly applied in healthcare to improve patient care and clinical outcomes. We previously developed an AI model using ICD-10 (International Classification of Diseases, Tenth Revision) codes with other clinical variables to predict in-hospital mortality among trauma patients from a nationwide database. This study aimed to externally validate the performance of the AI model. Validation was conducted using a multicenter retrospective cohort study design, analyzing patient data from January 2020 to December 2021. The study included trauma patients based on specific ICD-10 codes, with other clinical variables. The performance of the AI model was evaluated against conventional metrics, including the ISS, and the ICISS (ICD-based ISS), using sensitivity, specificity, accuracy, balanced accuracy, precision, F1-score, and area under the receiver operating characteristic curve (AUROC) analyses. Data from 4,439 patients were analyzed. The AI model demonstrated high overall performance, achieving an AUROC of 0.9448 and a balanced accuracy of 85.08%, thereby outperforming traditional scoring systems such as ISS, or ICISS. Furthermore, the model accurately predicted mortality across datasets from each hospital (AUROCs of 0.9234 and 0.9653, respectively) despite significant differences in hospital characteristics. In the subset of patients with ISS < 9, the model showed a robust AUROC of 0.9043, indicating its effectiveness in predicting mortality, even in cases with lower-severity injuries. For patients with ISSs ≥ 9, the model maintained high sensitivity (93.60%) and balanced accuracy (77.08%), proving its reliability in more severe injury cases. External validation demonstrated the AI model's high predictive accuracy and reliability in assessing in-hospital mortality risk among trauma patients across different injury severities and heterogeneous cohorts. These findings support the model's potential integration into emergency departments and offer a significant tool for enhancing patient triage and treatment protocols.