Patient Triage and Guidance in Emergency Departments Using Large Language Models: Multimetric Study.

Area of Science:

• Artificial Intelligence in Healthcare
• Clinical Decision Support Systems
• Emergency Medicine Technology

Background:

• Emergency departments (EDs) face significant challenges including overcrowding and staff shortages.
• Efficient patient triage and departmental guidance are crucial for managing ED pressures.
• Large language models (LLMs) like ChatGPT offer potential solutions for improving emergency care processes.

Purpose of the Study:

• To assess the accuracy and consistency of GPT-4 based ChatGPT models (GPT-4o, GPT-4-Turbo) for Modified Early Warning Score (MEWS) triage.
• To evaluate GPT-4o's accuracy in outpatient department selection using simulated patient scenarios.
• To determine the feasibility of LLMs in enhancing emergency department workflow.

Main Methods:

• A two-phase experimental study utilizing simulated patient scenarios.
• Phase 1: Evaluated MEWS triage accuracy of GPT-4o and GPT-4-Turbo using 1854 scenarios, assessing impact of prompt engineering.
• Phase 2: Assessed GPT-4o's outpatient department guidance accuracy with 264 scenarios from the Chinese Medical Case Repository.

Main Results:

• Prompt engineering improved ChatGPT's MEWS triage accuracy, with GPT-4-Turbo achieving 100% accuracy compared to GPT-4o's 96.2%.
• GPT-4o demonstrated 92.63% accuracy in outpatient department guidance, with highest accuracy in internal medicine (93.51%).
• GPT-4o showed superior emotional responsiveness, a key trait for patient-facing applications.

Conclusions:

• ChatGPT models show significant potential for supporting patient triage and outpatient guidance in ED settings.
• GPT-4-Turbo is more adaptable to prompt engineering for triage, while GPT-4o excels in patient interaction.
• Further research into real-world implementation is needed to optimize clinical integration of LLMs in emergency care.