Leveraging Large Language Models to Enhance Patient Educational Resources in Rhinology

Area of Science:

• Medical education
• Artificial intelligence in healthcare
• Health literacy

Background:

• Patient education materials (PEMs) on rhinologic conditions and procedures from the American Rhinologic Society (ARS) were evaluated.
• Assessing the readability of existing medical information is crucial for patient comprehension.

Purpose of the Study:

• To compare the readability of ARS-generated PEMs with those created by large language models (LLMs).
• To determine if LLMs can simplify complex medical information to a sixth-grade reading level.

Main Methods:

• Forty-one ARS PEMs were analyzed for readability using Flesch Kincaid Grade Level (FKGL) and Flesch Kincaid Reading Ease (FKRE).
• Three LLMs (ChatGPT 4.0, Google Gemini, Microsoft Copilot) translated ARS PEMs to a sixth-grade reading level.
• Readability scores of original and AI-generated PEMs were compared.

Main Results:

• A total of 164 PEMs were assessed, with 123 generated by LLMs.
• AI-generated PEMs showed significantly improved readability (mean FKGL 8.6) compared to original ARS PEMs (mean FKGL 10.28).
• Google Gemini yielded the most readable content (mean FKGL 7.5, FKRE 65.5).

Conclusions:

• LLMs enhance the readability of medical patient education materials, improving accessibility for diverse populations.
• While LLMs show promise, caution is advised when using AI-generated content for specific conditions like rhinology.
• Further research is needed to ensure the accuracy and safety of LLM-generated medical information.