Patient-Friendly Discharge Summaries in Korea Based on ChatGPT: Software Development and Validation

Area of Science:

• Artificial Intelligence in Healthcare
• Natural Language Processing
• Medical Informatics

Background:

• Patient-friendly discharge summaries improve comprehension and satisfaction.
• Generating these summaries can increase medical staff workload.
• Large language models (LLMs) offer a potential solution to automate summary generation.

Purpose of the Study:

• To develop and validate software using LLMs for generating patient-friendly discharge summaries.
• To compare the effectiveness of different prompting methods (Zero-shot, One-shot, Few-shot) for summary generation.
• To assess the quality and factuality of LLM-generated summaries.

Main Methods:

• Developed and tested LLM software on 100 discharge summaries (myocardial infarction and general surgery).
• Generated three summaries per document using Zero-shot, One-shot, and Few-shot prompting.
• Evaluated summaries using a 5-point Likert scale for factuality, comprehensiveness, usability, ease, and fluency.

Main Results:

• Few-shot and One-shot prompts significantly outperformed Zero-shot prompts (mean scores 4.19 and 4.11 vs. 3.73).
• 77.0% (Few-shot) and 70.0% (One-shot) of summaries scored ≥ 4, compared to 32.0% for Zero-shot.
• Factuality scores were high across all methods, with Few-shot and One-shot showing excellent results (≥4.19).

Conclusions:

• LLMs, particularly with Few-shot prompts, can generate acceptable patient-friendly discharge summaries.
• This technology shows promise for enhancing patient-centered care, especially for Korean patients.
• The study validates the potential of AI in improving medical communication efficiency and quality.