Assessing large language models as assistive tools in selecting first trial lens parameters for orthokeratology

Large language models (LLMs) have the potential to be powerful tools in optometry. Orthokeratology is widely used in clinical interventions for myopia control. This study aims to evaluate the performance of LLMs as assistive tools in the CRT-related orthokeratology fitting workflow.

This retrospective analysis used four LLMs (GPT-4o, GPT-o3, GPT-4.1 and Claude 3.7 Sonnet) to analyze refractive error cases and get responses regarding the parameters of the first trial lens. Subjective evaluation includes the accuracy and overall quality of the answers provided, and objective evaluation focuses on differences in the parameters of the first trial lens.

GQS and accuracy differed across models [χ²(3) = 39.85, p < 0.001; Kendall's W = 0.148]. GPT-o3 and GPT-4o showed the strongest overall performance on the complete response (GQS: 4.66 ± 0.48 vs. 4.47 ± 0.5, Good ratings: 83.3% vs. 76.7%), For first trial lens parameters, feasibility errors decreased across the two correction rounds, LLM outputs showed tendencies concentrated in key fitting parameters, particularly a smaller BC radius (mm) and a larger RZD, while Bland-Altman analyses indicated that most observations lay within the 95% limits of agreement.

LLMs may support routine CRT-related decision support. However, first trial-lens parameter selection required feasibility constraints and clinician verification, with systematic parameter bias mainly involving BC and RZD.