Evaluating an artificial intelligence scribe for clinical documentation

Artificial intelligence (AI)-based ambient listening technology can perform scribing, which can reduce burnout and lower the amount of time spent closing charts outside of working hours. Our objective was to understand the habits of clinicians using an AI scribe tool and its impact on perceptions of burnout, mental demand, and time in the electronic health record (EHR).

We recruited ten primary care physicians from the top 20% of electronic medical record time outside scheduled hours from an academic health system. Participants completed a baseline survey including Mini Z 2.0 for burnout and NASA Task Load Index to assess perceived burden, followed by training and installation of the tool. After using the tool for four weeks, we collected the same measures, and physicians participated in a semistructured interview. Quantitative and qualitative data were mixed and interpreted.

Physicians used the tool for a median of 61 encounters (range: 1-181). The only significant pre-post change in any measure was characters typed (decreased by 15,398; p = 0.03). Interviews revealed facilitators and barriers to usage. Facilitators included the tool's accuracy and optimism that it could contribute to reduced workload. Barriers were the tool's lack of EHR integration, inability to identify multiple people in the consultation, and difficulties handling complex health issues.

We found varied results after using the AI scribe tool. Characters typed decreased, but no other change was significant. Physicians were hopeful the tool could eventually reduce workload, but implementation barriers must be addressed before it is widely accepted.