Related Experiment Video
Updated: May 8, 2026
Assessing Early Stage Open-Angle Glaucoma in Patients by Isolated-Check Visual Evoked Potential
Published on: May 25, 2020
Assessing the appropriateness and completeness of ChatGPT-4's AI-generated responses for queries related to diabetic
Brughanya Subramanian1, Ramachandran Rajalakshmi2, Sobha Sivaprasad3
1Department of Ophthalmology, Shri Bhagwan Mahavir Vitreoretinal Services, SankaraNethralaya, Chennai, Tamil Nadu, India.
Objective:
To evaluate the appropriateness of responses generated by an online chat-based artificial intelligence (AI) model for diabetic retinopathy (DR) related questions.
Design:
Cross-sectional study.
Methods:
A set of 20 questions framed from the patient's perspective addressing DR-related queries, such as the definition of disease, symptoms, prevention methods, treatment options, diagnostic methods, visual impact, and complications, were formulated for input into ChatGPT-4. Peer-reviewed, literature-based answers were collected from popular search engines for the selected questions and three retinal experts reviewed the responses. An inter-human agreement was analyzed for consensus expert responses and also between experts. The answers generated by the AI model were compared with those provided by the experts. The experts rated the response generated by ChatGPT-4 on a scale of 0-5 for appropriateness and completeness.
Results:
The answers provided by ChatGPT-4 were appropriate and complete for most of the DR-related questions. The response to questions on the adverse effects of laser photocoagulation therapy and compliance to treatment was not perfectly complete. The average rating given by the three retina expert evaluators was 4.84 for appropriateness and 4.38 for completeness of answers provided by the AI model. This corresponds to an overall 96.8% agreement among the experts for appropriateness and 87.6% for completeness regarding AI-generated answers.
Conclusion:
ChatGPT-4 exhibits a high level of accuracy in generating appropriate responses for a range of questions in DR. However, there is a need to improvise the model to generate complete answers for certain DR-related topics.
