Evaluating Large Language Models to Improve Spanish Patient Education on Childhood Glaucoma

To evaluate the effectiveness of ChatGPT-4o (OpenAI) in enhancing the readability and quality of Spanish patient educational materials (PEMs) on childhood glaucoma, and to determine whether they are written above the recommended 6th-grade reading level (per American Medical Association recommendation).

This cross-sectional comparative study analyzed 10 original Spanish-language PEMs on "glaucoma infantil." Each was inputted into ChatGPT-4o using a Spanish-language prompt requesting improved readability to a score of 70 or greater on the INFLESZ scale (adaptation of the Flesch-Szigriszt Index for Spanish-language texts), which is equivalent to a 4th- to 6th-grade reading level. Original and revised PEMs were compared using the INFLESZ scale. Additionally, three native Spanish-speaking ophthalmologists assessed each PEM's quality using a published 15-point Likert scale evaluating helpfulness, truthfulness, and harmlessness.

Original PEMs had a mean INFLESZ score of 56.6 ± 4.9 (8th-grade level). ChatGPT-4o significantly improved readability to 69.6 ± 2.3 (4th- to 6th-grade level, P < .001). Revised PEMs had fewer syllables, words, and complex words without a significant difference in sentence count. Quality remained high across both groups (median score = 15, P = .74), and no hallucinations were observed in artificial intelligence-generated text.

ChatGPT-4o can be used as a supplemental tool by health care professionals to improve the readability of existing Spanish PEMs on childhood glaucoma, without sacrificing the quality, accuracy, or reliability of their content.