Can LLMs simplify operative notes? A comparative analysis in otorhinolaryngology.

Area of Science:

• Medical Informatics
• Natural Language Processing

Background:

• Operative notes are vital for surgical documentation but often contain complex medical jargon.
• This complexity hinders understanding for patients and non-specialist healthcare professionals.
• Large Language Models (LLMs) present a potential solution for simplifying these medical texts.

Purpose of the Study:

• To evaluate the effectiveness of six LLMs in simplifying otolaryngology operative notes.
• To compare the readability, clinical accuracy, and clarity of LLM-generated simplified texts.
• To analyze LLM performance across different otolaryngologic subspecialties.

Main Methods:

• Thirty-nine fictional otolaryngologic operative notes were simplified using six LLMs.
• Readability was assessed using eight distinct metrics.
• Expert physicians evaluated the simplified notes for medical accuracy and comprehensibility.

Main Results:

• GPT-4o, Gemini, and DeepSeek produced the most readable simplified notes.
• Claude 3.7 generated the most complex outputs.
• GPT-4 demonstrated the highest medical accuracy, while GPT-4o excelled in clarity.
• LLM performance varied across clinical subgroups (rhinology, otology, head and neck surgery).

Conclusions:

• LLMs are effective tools for enhancing the accessibility of medical texts.
• The choice of LLM should align with the intended audience and clinical context.
• All LLM-generated medical content requires verification by qualified medical experts.
• Validated LLM use can significantly advance health communication.