Generative Artificial Intelligence Summaries to Facilitate Emergency Department Handoff.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Communication

Background:

• Emergency Department (ED) to inpatient handoffs are critical but prone to errors.
• Generative Artificial Intelligence (AI) offers potential for concise clinical data summarization.
• AI may improve the accuracy and efficiency of ED handoffs.

Purpose of the Study:

• Evaluate the accuracy, clinical utility, and safety of AI-generated ED handoff summaries.
• Identify patient and visit factors influencing AI summary effectiveness.
• Characterize AI error patterns to guide implementation.

Main Methods:

• Exploratory study at an urban academic ED (Feb-Apr 2024).
• HIPAA-compliant GPT-4 model generated ED-to-inpatient summaries using the IPASS framework.
• ED providers assessed summary accuracy, usefulness, and safety via on-shift surveys.

Main Results:

• Median quality and usefulness scores were 4/5 across 50 cases.
• Safety concerns (omissions, mischaracterizations) occurred in 6% of cases.
• Consultation status impacted usefulness; clinicians noted omissions and disagreements on patient stability.

Conclusions:

• The AI provider-in-the-loop model demonstrated clinical acceptability.
• Areas for refinement include addressing data omissions and characterization accuracy.
• Future research should include recipient feedback and clinical outcomes for AI optimization.