Can artificial intelligence match refractive surgeons?: A comparative analysis of refractive surgery candidate selection

This study aimed to evaluate the accuracy of 2 large language models (LLMs) - ChatGPT-5.0 and Gemini Flash 2.5 - in assessing refractive-surgery eligibility and recommending appropriate procedures, using refractive-surgeon consensus as the reference standard. This retrospective, single-center study included the right eyes of 40 consecutive patients evaluated for refractive surgery at Başkent University Hospital (Istanbul, Türkiye). To avoid inter-eye correlation, only the right eye of each patient was included. Standardised refractive and corneal tomographic data obtained using the Pentacam HR were independently reviewed by 2 refractive surgeons and by the 2 LLMs. Each model received identical English-language prompts once per eye. Outputs were classified as "eligible," "borderline," or "ineligible"; for all binary statistical analyses, borderline assessments were grouped with the ineligible category. Recommended surgical procedures were recorded separately, with more than 2 procedure permitted for the same eye. Agreement between experts and between each LLM and expert consensus was assessed using Gwet's agreement coefficient 1 (AC1) coefficient. Inter-expert agreement was good (AC1 = 0.78). Compared with expert consensus, ChatGPT-5.0 demonstrated very good agreement (AC1 = 0.92), outperforming Gemini Flash 2.5, which showed good agreement (AC1 = 0.78). Both LLMs accurately identified most "eligible" eyes; however, specificity and negative predictive values exhibited wide confidence intervals due to the limited number of "ineligible" cases. ChatGPT-5.0 showed higher sensitivity (92.1%) and negative predictive value (40.0%) than Gemini Flash 2.5 (81.6% and 22.2%, respectively). Notably, neither model recommended implantable collamer lens surgery, even when clinically appropriate. ChatGPT-5.0 demonstrated very good agreement with expert consensus and higher agreement than Gemini Flash 2.5 in refractive-surgery eligibility assessment. Although both LLMs show potential as adjunctive decision-support tools, the omission of large language model recommendations and prompt-dependent variability limit their suitability for autonomous clinical use. Further optimization using comprehensive refractive-surgery datasets is required to improve clinical reliability.