Human vs. artificial intelligence in medical charting: a comparative study in the simulated emergency medicine...

The increasing administrative burden associated with electronic health records has contributed to reduced efficiency and rising burnout among physicians, particularly in high-volume environments like Emergency Departments (EDs). Ambient artificial intelligence (AI) scribes offer a potential solution by generating clinical documentation in real-time using ambient listening. This study aimed to evaluate the efficiency of an AI scribe compared to traditional voice dictation in simulated ED encounters, assess user satisfaction, and analyze the types and extent of edits required in AI-generated notes.

We conducted a mixed methods simulated study using standardized patient encounters with Emergency Medicine physicians and residents at the Ottawa Hospital. Participants completed three scripted clinical scenarios. Each interaction was recorded using an AI scribe (Mutuo) and participants subsequently documented the encounter using Dragon voice dictation. The AI-generated note was then edited by the participant, with time tracked for both dictation and editing. Quantitative analysis included a paired t-test for time comparison and Wilcoxon rank sum test for note preference. Qualitative feedback was collected via structured surveys and open-ended questions.

Sixteen participants completed the study, with 81% being attending physicians. Mean dictation time was 2:05 min compared to 1:15 min for AI note editing, representing a 39% time savings (p < 0.0001). Of 240 total note sections, 69% required no edits; edits were most common in the "History" and "Plan" sections. Most edits (76%) were minor, involving only a few words or a single sentence. Sixty-three percent of participants preferred the AI-generated note (p < .001).

In a simulated ED setting, ambient AI scribe technology significantly reduced documentation time with minimal required editing. These findings support the potential clinical utility of AI scribes in improving physician efficiency, warranting further evaluation in real-world ED environments.