Development and evaluation of a clinical note summarization system using large language models.

Area of Science:

• Artificial Intelligence in Healthcare
• Natural Language Processing
• Clinical Informatics

Background:

• Clinical notes contain vital patient hospitalization data but are challenging to evaluate due to volume and complexity.
• Accurate summarization of clinical notes is crucial for effective clinical decision-making.
• Large language models (LLMs) offer a promising approach for generating coherent and contextually relevant clinical text.

Purpose of the Study:

• To develop a discharge summary system utilizing large language models.
• To ensure the system meets the practical needs and experiences of end-users, including physicians and patients.
• To establish a robust evaluation framework for assessing the LLM's performance.

Main Methods:

• Conducted online surveys and interviews with physicians and patients to gather user feedback.
• Developed a rating system to evaluate prompt effectiveness.
• Compared LLM-generated outputs against human assessments as benchmarks.

Main Results:

• The LLM demonstrated diagnostic interpretation accuracy approaching human levels.
• The system shows potential for assisting healthcare professionals with routine documentation tasks.
• User feedback was incorporated to refine the system's practical utility.

Conclusions:

• LLMs show significant potential for enhancing clinical settings.
• This technology can streamline healthcare documentation processes.
• LLMs open new avenues for decision-making support in healthcare.