Implementing Generative AI to Enhance Patient Education on Retinopathy of Prematurity

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Patient Education

Background:

• Patient education materials (PEMs) are crucial for informing parents about complex conditions like retinopathy of prematurity (ROP).
• Ensuring PEMs are accurate, understandable, and accessible is a significant challenge in pediatric ophthalmology.
• Large language models (LLMs) offer potential solutions for efficiently generating and refining health information.

Purpose of the Study:

• To assess the capability of different LLMs in creating patient education materials (PEMs) specifically for retinopathy of prematurity (ROP).
• To compare the efficacy of ChatGPT-3.5, ChatGPT-4, and Gemini in generating ROP-related educational content.
• To evaluate the readability, quality, and accuracy of LLM-generated PEMs.

Main Methods:

• Three LLMs (ChatGPT-3.5, ChatGPT-4, Gemini) were prompted to generate novel PEMs on ROP, create PEMs at a 6th-grade reading level, and revise existing PEMs for improved readability.
• Readability was assessed using the Simple Measure of Gobbledygook (SMOG) and Flesch-Kincaid Grade Level (FKGL) formulas.
• Quality was evaluated using the Patient Education Materials Assessment Tool (PEMAT) and DISCERN criteria, while accuracy was measured using a Likert Misinformation Scale.

Main Results:

• LLM-generated PEMs demonstrated high quality, understandability (PEMAT-U ≥ 70%), and accuracy (Likert = 1).
• Materials generated specifically for a 6th-grade reading level (Prompt B) were significantly more readable than novel generations (Prompt A).
• ChatGPT-4 and Gemini successfully reduced the reading level of existing PEMs; however, only ChatGPT-4 maintained high quality and reliability.

Conclusions:

• LLMs, particularly ChatGPT-4, show significant promise as tools for automating the creation of high-quality, readable patient education materials on ROP.
• These AI models can assist in producing supplementary resources for parents, enhancing their understanding of retinopathy of prematurity.
• Further research can explore integrating LLM-generated content into standard clinical practice for pediatric eye conditions.