Feasibility of Tailoring Artificial Intelligence-Assisted Ambient Scribes for Intensive Care Unit Rounds: Algorithm...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Documentation Improvement

Background:

• Physician burnout is exacerbated by increasing documentation demands.
• Current solutions for reducing documentation burden in inpatient and intensive care unit (ICU) settings are limited.
• Artificial intelligence (AI) offers a potential solution for clinical documentation challenges.

Purpose of the Study:

• To explore the feasibility of AI-assisted ambient scribes for capturing interprofessional ICU rounds.
• To synthesize round transcripts into a single document to enhance documentation efficiency and clinician satisfaction.
• To evaluate the effectiveness of customized prompts for large language models (LLMs) in generating daily progress notes from simulated ICU cases.

Main Methods:

• A two-phase pilot study involving simulated ICU cases.
• Phase 1: Iterative prompt refinement for LLMs using audio transcripts, selecting the best model (M1).
• Phase 2: Evaluation of M1 and an upgraded model (M6) using refined prompts, assessing accuracy, error rates, error severity (Harm scale), and usability (Physician Documentation Quality Instrument).

Main Results:

• Iterative prompt improvements enhanced LLM accuracy and reduced errors.
• Model M6 achieved 80% accuracy, significantly outperforming M1 (69%).
• Errors were predominantly omissions; both models showed low error severity, with minimal harm potential. M6 also showed better usability scores.

Conclusions:

• AI-assisted scribes are feasible for ICU documentation, improving efficiency and satisfaction.
• Prompt engineering and LLM advancements are key drivers of AI performance in clinical settings.
• This study provides a foundation for further research into AI applications to optimize healthcare documentation.