Performance of a novel medical artificial intelligence large language model on supporting decision-making for...

Large language models (LLMs) are being explored for disease prediction and diagnosis; however, their efficacy for early sepsis identification in emergency departments (EDs) remains unexplored. This study aims to evaluate MedGo, a novel medical LLM, as a decision-support tool for clinicians managing patients with suspected sepsis.

This retrospective study included anonymized medical records of 203 patients (mean age 79.9±10.2 years) with confirmed sepsis from a tertiary hospital ED between January 2023 and January 2024. MedGo performance across nine sepsis-related assessment tasks was compared with that of two junior (<3 years of experience) and two senior (>10 years of experience) ED physicians. Assessments were scored on a 5-point Likert scale for accuracy, comprehensiveness, readability, and case-analysis skills.

MedGo demonstrated diagnostic performance comparable to that of senior physicians across most metrics, achieving a median Likert score of 4 in accuracy, comprehensiveness, and readability. MedGo significantly outperformed junior physicians (P<0.001 for accuracy and case-analysis skills). MedGo assistance significantly enhanced both junior (P<0.001) and senior (P<0.05) physicians' diagnostic accuracy. Notably, MedGo-assisted junior physicians achieved accuracy levels comparable to those of unassisted senior physicians. MedGo maintained consistent performance across varying sepsis severities.

MedGo shows significant diagnostic efficacy for sepsis and effectively supports clinicians in the ED, particularly enhancing junior physicians' performance. Our study highlights the potential of MedGo as a valuable decision-support tool for sepsis management, paving the way for specialized sepsis AI models.