Evaluating the efficacy of large language models in predicting intensive care unit admission needs

Timely identification and transfer of critically ill patients to intensive care units (ICUs) are crucial to reducing morbidity and mortality. Delayed ICU admission is linked to higher mortality, emphasizing the need for efficient prediction systems. Artificial intelligence (AI) has shown promise in enhancing clinical decision-making. This study evaluates the efficacy of ChatGPT and Gemini AI models in predicting ICU admission needs.

This retrospective observational study analyzed data from 8043 ICU consultation cases in a tertiary hospital between January 2020 and June 2024. Clinical parameters included medication use, consultation details, ECG findings, imaging results, comorbidities, and laboratory values. Preprocessed and anonymized data were analyzed using ChatGPT and Gemini, with performance assessed through accuracy, Kappa statistic, Pearson Chi-square, and logistic regression.

ChatGPT demonstrated strong predictive performance, achieving an accuracy of 93.8% and a Kappa statistic of 0.802, indicating substantial agreement with anesthesiologists' ICU decisions. Its predictions showed a highly significant association with actual clinical outcomes (Pearson Chi-Square = 5293.310, p < 0.001). Sensitivity and specificity analyses further confirmed ChatGPT's balanced performance in identifying both ICU and non-ICU patients. In contrast, Gemini showed more modest agreement with clinical decisions, with a Kappa value of 0.396, reflecting lower consistency despite producing correct classifications in a portion of cases.

ChatGPT demonstrated superior accuracy and reliability in predicting ICU needs compared to Gemini. Early identification of critically ill patients enables timely interventions, potentially reducing morbidity and mortality. Future studies should explore real-time AI integration into clinical workflows and validate findings through prospective research.