Enhancing the Readability of Online Patient Education Materials Using Large Language Models: Cross-Sectional Study

Area of Science:

• Health Informatics
• Natural Language Processing in Healthcare
• Patient Education

Background:

• Patient education materials (PEMs) are crucial for patient empowerment but often written above a sixth-grade reading level.
• This inaccessibility limits understanding for a significant portion of the patient population.
• Large language models (LLMs) offer a potential solution for simplifying complex medical information.

Purpose of the Study:

• To evaluate the effectiveness of three LLMs (ChatGPT, Gemini, Claude) in optimizing the readability of online PEMs.
• To assess if LLM simplification compromises the accuracy and understandability of the educational content.
• To determine if LLMs can reduce PEMs to the recommended sixth-grade reading level.

Main Methods:

• A cross-sectional study analyzed 60 randomly selected online PEMs from three health websites.
• LLMs were prompted to simplify the reading level of the selected PEMs.
• Readability was assessed using Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog Index, and Simple Measure of Gobbledygook (SMOG) indices. Accuracy and understandability (PEMAT-U) were also evaluated.

Main Results:

• Original PEMs had mean grade level scores above the sixth-grade recommendation (AHA: 10.7, ACS: 10.0, ASA: 9.6).
• LLM simplification significantly improved readability across all sites (ChatGPT to 7.6, Gemini to 6.6, Claude to 5.6, P<.001).
• Word counts were reduced, and baseline understandability was maintained; however, 3.3% of Gemini and Claude outputs contained inaccuracies.

Conclusions:

• LLMs show significant potential to enhance the readability and accessibility of online patient education materials.
• Maintaining accuracy and understandability is achievable, but model performance varies.
• Human oversight is critical to review LLM-generated content for any inaccuracies before patient dissemination.