Generative artificial intelligence for inpatient documentation summarization: mixed-methods quality assessment and...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Documentation

Background:

• Ongoing hospital care generates vast amounts of clinical data.
• Efficient summarization of patient information is crucial for effective clinical decision-making.
• Current methods for summarizing patient data can be time-consuming and prone to errors.

Purpose of the Study:

• To evaluate the quality and adoption of a large language model (LLM)-based summarization tool for hospital care.
• To assess user perceptions of the tool's accuracy, comprehensibility, and usefulness.
• To analyze utilization patterns and gather qualitative feedback on the AI summarization tool.

Main Methods:

• A pragmatic mixed-methods quality assessment was conducted at three academic hospitals.
• An AI summarization tool was developed to provide patient summaries, including "Patient Story" and "Recent Notes".
• The Provider Documentation Summarization Quality Instrument (PDSQI-9), utilization analytics, and qualitative feedback were used for evaluation.

Main Results:

• Users rated the tool favorably across all PDSQI-9 domains, with a combined average score of 4.68/5.
• Utilization metrics indicated strong adoption and frequent use of the summarization tool.
• Qualitative feedback highlighted cognitive offloading benefits but also noted a need for improved specialty-specific summarization and data cross-referencing.

Conclusions:

• LLM-based summarization tools show transformative potential for generating succinct and verifiable summaries in healthcare.
• The evaluated AI tool demonstrated high perceived quality, a favorable safety profile, and sustained utilization among clinicians.
• Further development should focus on enhancing specialty-specific details and integrating current-state data more effectively.