Prediction of refractive error in adolescents using a multimodal large language model

This study aims to develop and evaluate a multimodal large language model (LLM) for predicting refractive error (diopter) in adolescents by integrating fundus images with clinical and demographic data. The goal is to demonstrate how such a model can serve as a digital health tool for early screening and personalized management of refractive conditions in clinical and remote health settings.

A dataset of 16,226 annotated records from adolescents aged 2 to 18 years was used. A vision-language foundation model based on Qwen2.5-VL was fine-tuned using supervised learning, incorporating both imaging and clinical data to predict spherical equivalent (SE). The model was trained on a randomly split dataset, and performance was evaluated using mean absolute error (MAE), root mean square error (RMSE), coefficient of determination (R ²) and the Pearson correlation coefficient.

The model achieved a mean absolute error of 0.647 diopters and showed strong predictive performance across most refractive subgroups. The incorporation of multimodal data significantly outperformed single-modality models. Visualization analyses confirmed that the model's predictions are clinically interpretable and reliable.

This multimodal LLM demonstrates the potential of digital health technologies to enhance refractive error prediction in adolescents. Its integration into digital health platforms can provide a non-invasive, scalable solution for early myopia screening, making it an invaluable tool for clinicians in both remote and urban settings.