Adapted large language models can outperform medical experts in clinical text summarization

Area of Science:

• Medical Informatics
• Natural Language Processing
• Artificial Intelligence in Healthcare

Background:

• Clinicians face significant time burdens analyzing electronic health records.
• Large language models (LLMs) show potential for natural language processing (NLP) but their clinical summarization effectiveness is unproven.

Purpose of the Study:

• To evaluate the effectiveness of adapted LLMs across diverse clinical text summarization tasks.
• To compare LLM-generated summaries against those produced by medical experts.

Main Methods:

• Adaptation methods were applied to eight LLMs for four tasks: radiology reports, patient questions, progress notes, and doctor-patient dialogue.
• Quantitative NLP metrics and a reader study with 10 physicians assessed summary quality (completeness, correctness, conciseness).

Main Results:

• Quantitative NLP metrics revealed trade-offs between LLM models and adaptation methods.
• Physician evaluations found LLM-generated summaries equivalent (45%) or superior (36%) to expert summaries.
• Safety analysis identified error types and potential medical harm for both LLMs and experts.

Conclusions:

• Adapted LLMs demonstrate strong performance in clinical text summarization, often matching or exceeding medical expert quality.
• Integrating LLMs into clinical workflows could significantly reduce documentation burden and enhance patient care focus.