Large language model discharge summary preparation using real-world electronic medical record data shows promise

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Documentation

Background:

• Discharge summary preparation is a time-consuming administrative task for clinicians.
• Large Language Models (LLMs) are emerging tools with potential applications in healthcare.

Purpose of the Study:

• To evaluate the efficacy of two distinct LLMs in generating discharge summaries.
• To assess the performance of LLM-generated discharge summaries using a validated scoring metric.

Main Methods:

• Two LLMs (llama3:instruct and llama3:70b) were utilized to generate discharge summaries from real clinical documentation.
• A validated discharge summary scoring metric was employed to assess the quality of the generated summaries.

Main Results:

• Both LLMs demonstrated comparable performance in generating discharge summaries.
• The llama3:instruct model achieved a mean score of 19.1/31 (SD: 2.42).
• The llama3:70b model achieved a mean score of 19.2/31 (SD: 3.48).

Conclusions:

• LLMs show comparable efficacy in generating discharge summaries.
• The use of LLMs in discharge summary generation may alleviate clinical administrative workload.