Quality and Readability of Patient Educational Materials Generated by ChatGPT-4o for Pediatric Ophthalmologic...

To evaluate the quality and readability of ChatGPT-4o-generated (ChatGPT) (OpenAI) patient education materials (PEMs) about pediatric ophthalmologic surgical procedures and compare these to PEMs from the American Association for Pediatric Ophthalmology and Strabismus (AAPOS) website.

The authors prompted ChatGPT-4o to provide PEMs on four procedures-strabismus surgery without adjustable sutures, strabismus surgery with adjustable sutures, pediatric cataract surgery, and nasolacrimal duct probing. The prompt requested responses at a 6th grade level in both Spanish and English. English ChatGPT responses were compared to AAPOS PEMs on quality (using the Quality of Generated Language Outputs for Patients [QGLOP] scale) and readability. English and Spanish ChatGPT responses were also compared on quality and readability.

Based on average scores from the four procedures, AAPOS PEMs were superior to English Chat-GPT responses on the accuracy, currency, and tone subscales of the QGLOP score (4.0 ± 0 vs 2.79 ± 0.79, P = .0021; 3.79 ± 0.26 vs 3.38 ± 0.71, P = .033; 4.0 ± 0 vs 3.42 ± 0.69, P = .042, respectively). There was no significant difference in readability between AAPOS PEMs and English ChatGPT responses. English and Spanish ChatGPT responses did not significantly differ on quality or readability.

ChatGPT-4o-generated PEMs on pediatric ophthalmologic surgical conditions are currently inferior in quality to PEMs on the AAPOS website. However, because ChatGPT is continually being updated and trained, this study should be repeated in the future to determine whether metrics improve over time.