Utility of ChatGPT-4o in Creating Patient Handouts in Ophthalmology: A Comparison With American Academy of...

This study compares the readability and quality of ChatGPT-4o-generated and American Academy of Ophthalmology (AAO) patient handouts in English and Spanish, as AAO materials are expert developed but not publicly accessible.

Ten AAO handouts on common ocular conditions were obtained in English and Spanish. ChatGPT-4o was queried to create handouts at an 8th-grade reading level of comparable length. English readability was assessed with Simple Measure of Gobbledygook (SMOG) and Flesch-Kincaid Reading Ease, and Spanish readability was assessed with five validated metrics, including the Szigriszt-Pazos Perspicuity Index (SPPI) and the Flesch-Szigriszt Index (INFLESZ). Grade levels were measured by the Flesch-Kincaid Grade Level and Crawford-Nivel-de-Grado. Three ophthalmologist graders per language assessed content with the Quality of Generated Language Outputs for Patients (QGLOP), indicated their preferred handout, and attempted to identify its source.

ChatGPT-4o handouts had English readability similar to that of AAO (SMOG 7.86 ± 0.53 vs. 7.49 ± 0.76; P = 0.35) but significantly lower Spanish SPPI scores (60.26 ± 4.47 vs. 64.82 ± 2.97; P = 0.04). Grade levels were comparable. English QGLOP scores were similar (14.73 vs. 14.87; P = 0.97), but Spanish ChatGPT handouts scored higher in all categories (15.83 vs. 14.23; P < 0.001). ChatGPT-4o handouts were correctly identified by 17 of 30 in English (56.7%) and by 3 of 30 in Spanish (10.0%). They were favored 23 out of 30 times in English (76.7%) and 29 out of 30 times in Spanish (96.7%).

ChatGPT-generated materials matched or exceeded AAO handouts in readability and content quality, with Spanish versions being most preferred.

Integrating artificial intelligence-generated patient education materials into ophthalmic care can enhance health literacy for multilingual patient populations.