EyeGraph: A Multi-granularity Medical Vision-Language Foundation Model with Knowledge Graph Reasoning for Ophthalmic...

The exponential growth of large vision-language models (VLMs) has created significant opportunities for advancing multimodal artificial intelligence in specialized domains such as ophthalmology. However, progress in ophthalmic VLMs remains limited, primarily due to the lack of large-scale, high-quality image-text pairs. Moreover, existing VLMs frequently generate hallucinations, leading to uncertainty and unreliability that are unacceptable in clinical practice. To address the data scarcity issue, we introduce IT-Retinal, a dedicated ophthalmic Vision Question Answering (VQA) dataset that providesmultigranularity tasks and high-quality long-form textual content in image-text pairs. To mitigate hallucinations, we propose EyeGraph, a framework for ophthalmic VLMs that consists of two training stages, targeted pre-training and fine-tuning, and one inference stage that integrates Graph of Thought (GoT) and Think on Graph (ToG) reasoning, referred to as G2T. This design enables EyeGraph to incorporate structured external knowledge, constrain the output space, and produce more clinically consistent results. Extensive experiments demonstrate that EyeGraph achieves state-of-the-art performance across seven clinically relevant benchmarks, spanning disease diagnosis, structural detection and localization, and report generation. Notably, the proposed G2T reasoning module consistently improves the factual consistency and reliability of ophthalmic report generation. All source code, along with an interactive demo, is publicly available at: https://github.com/ZardCAI/EyeGraph.