Related Experiment Video
Updated: May 8, 2025
Author Spotlight: Unraveling the Molecular Mechanisms in PCO and Fibrosis Following Cataract Surgery
Published on: December 1, 2023
Conversational Guide for Cataract Surgery Complications: A Comparative Study of Surgeons versus Large Language
Sathishkumar Sundaramoorthy1, Vineet Ratra2, Vijay Shankar3
1Department of Vitreoretinal Diseases, Medical Research Foundation, Sankara Nethralaya, Chennai, India.
Purpose:
It is difficult to explain the complications of surgery to patients. Care has to be taken to convey the facts clearly and objectively while expressing concern for their wellbeing. This study compared responses from surgeons with responses from a large language model (LLM)-based chatbot.
Methods:
We presented 10 common scenarios of cataract surgery complications to seven senior surgeons and a chatbot. The responses were graded by two independent graders for comprehension, readability, and complexity of language using previously validated indices. The responses were analyzed for accuracy and completeness. Honesty and empathy were graded for both groups. Scores were averaged and tabulated.
Results:
The readability scores for the surgeons (10.64) were significantly less complex than the chatbot (12.54) (p < 0.001). The responses from the surgeons were shorter, whereas the chatbot tended to give more detailed answers. The average accuracy and completeness score of chatbot-generated conversations was 2.36 (0.55), which was similar to the surgeons' score of 2.58 (0.36) (p = 0.164). The responses from the chatbot were more generalized, lacking specific alternative measures. While empathy scores were higher for surgeons (1.81 vs. 1.20, p = 0.041), honesty scores showed no significant difference.
Conclusions:
The LLM-based chatbot gave a detailed description of the complication but was less specific about the alternative measures. The surgeons had a more in-depth understanding of the situation. The chatbot showed complete honesty but scored less for empathy. With more training using complex real-world scenarios and specialized ophthalmologic data, the chatbots could be used to assist the surgeons in counselling patients for postoperative complications.
