Artificial Intelligence Models for Predicting Triage in Emergency Departments: Seven-Month Retrospective Comparative...

Area of Science:

• Emergency Medicine
• Artificial Intelligence
• Clinical Informatics

Background:

• Triage errors in emergency departments (EDs) risk patient safety and resource allocation.
• Increasing patient volumes and staffing challenges highlight the need for AI in triage protocols.

Purpose of the Study:

• Develop and compare three AI models (NLP, LLM, JEPA) for predicting French Emergency Nurses Classification in Hospital (FRENCH) triage outcomes.
• Assess AI model performance against nurse triage and clinical expert consensus.

Main Methods:

• Retrospective analysis of 73,236 adult ED visits (June-December 2024) with complete audio/structured data.
• Developed TRIAGEMASTER (NLP), URGENTIAPARSE (LLM), and EMERGINET (JEPA) models.
• Evaluated concordance with gold-standard FRENCH triage using weighted κ, Spearman correlation, F1-score, AUC-ROC, MAE, and RMSE.

Main Results:

• URGENTIAPARSE (LLM) demonstrated superior performance (composite z=2.514) compared to EMERGINET (0.438), TRIAGEMASTER (-3.511), and nurse triage (-4.343).
• URGENTIAPARSE achieved high accuracy (F1=0.900, AUC=0.879, κ=0.800), but exhibited overfitting (training=1.0, validation≈0.5).
• TRIAGEMASTER and nurse triage performed poorly (F1=0.618, 0.303).

Conclusions:

• LLM-based URGENTIAPARSE showed highest accuracy for ED triage prediction but has limited clinical applicability due to overfitting and selection bias.
• Rigorous validation, bias mitigation, and safety evaluation are essential before AI triage system deployment.
• AI triage support systems show promise but require careful implementation to ensure patient safety and reliability.