Verifiable Summarization of Electronic Health Records Using Large Language Models to Support Chart Review

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Clinical Decision Support

Background:

• Information overload in electronic health records (EHRs) presents a significant challenge for clinicians, hindering efficient data synthesis and potentially delaying patient care.
• Heart failure is a leading cause of hospitalization, necessitating effective methods for managing complex patient information.

Purpose of the Study:

• To evaluate the efficacy of large language models (LLMs) in generating problem-based admission summaries from longitudinal clinical notes for heart failure patients.
• To assess the impact of LLM-generated summaries on clinician efficiency, accuracy, and workload compared to traditional chart review.

Main Methods:

• Development of an extract-then-abstract approach using disease-specific "summary bundles" to prioritize clinically relevant information from EHRs.
• Mixed-methods evaluation involving physicians answering patient-specific clinical questions using LLM-generated summaries versus standard chart review.

Main Results:

• Frequent use of LLM-generated summaries significantly accelerated questionnaire completion (p = 0.002), although overall time was not significantly reduced.
• Summary use maintained high accuracy in answering clinical questions (88.0% with summaries vs. 86.4% without).
• Physicians expressed strong likelihood of using summaries in practice, anticipating time savings, with varied preferences for summary format.

Conclusions:

• Large language models show promise in generating effective longitudinal summaries for EHRs, aiding clinicians in managing information overload.
• LLM-generated summaries have the potential to enhance clinician efficiency, reduce workload, and support informed decision-making in critical care settings.
• Customizable summary formats are crucial to align with individual clinician workflows and maximize the benefits of LLM summarization tools.