ED-triage-agent: a multi-agent framework for human-in-the-loop emergency triage

Emergency Department (ED) triage requires rapid assessment of acuity under time pressure with incomplete information. Existing AI triage systems require complete clinical data, including vital signs, before generating a prediction, and so cannot support the sequential, information-gathering reality of triage in which patients accumulate in queues before vital signs are collected. Pre-vital queue prioritisation, deciding which patients are seen next before vital signs exist, remains unaddressed.

To design and evaluate ED-Triage-Agent (ETA), a multi-agent large language model architecture that produces a provisional, ESI-scaled acuity signal from symptom-only data for queue prioritisation, and a criterion-linked ESI recommendation once vital signs are available.

ETA employs five specialised agents orchestrated via LangGraph, grounded in the ESI Implementation Handbook v4 via retrieval-augmented generation. Confidence weights and decision thresholds were calibrated on 30 standardised ESI Handbook Practice Cases; performance was evaluated on the external TRIAGEAGENT benchmark using exact-match accuracy, quadratic-weighted Cohen's κw, F1, under-triage and over-triage rates, and high-priority sensitivity, with 95% bootstrap confidence intervals. Single-model and multi-agent variants, with and without RAG, were compared against the full architecture on 30 standardised ESI Handbook Competency Cases.

On the TRIAGEAGENT benchmark, Phase 1 pre-vital prioritisation achieved 76.39% exact-match accuracy (κ_w = 0.8787) and 95.52% high-priority sensitivity. Phase 2 post-vital classification achieved 87.04% exact-match accuracy (κ_w = 0.9090), with 0.00% significant over-triage and 0.46% significant under-triage. Misclassifications were predominantly between adjacent ESI levels (97.22% within ± 1). The chain-of-thought baseline showed 10.00% significant under-triage on the Competency Cases, versus 0.00% for ETA.

This study demonstrates the technical feasibility of a two-phase multi-agent architecture for generating provisional, ESI-scaled risk signals under partial information, with performance on standardized cases comparable to previously reported benchmark results. Clinical utility, safety, and workflow integration remain to be established through prospective validation with real ED data and clinician users.