EyeRAG: graph retrieval-augmented generation for safe and accurate clinical dialogue in ophthalmology

Large language models promise to transform ophthalmic clinical communication but face challenges from factual inaccuracies (hallucinations) and limited domain knowledge. This study introduces EyeRAG, a guideline-grounded GraphRAG system for ophthalmic dialogue that integrates OphthaKG, a domain-specific knowledge graph, constructed exclusively from clinical guidelines. We evaluated EyeRAG across 120 clinical scenarios (40 each for glaucoma, diabetic retinopathy, and cataract), six LLMs (GPT-4o, Gemini 2.5 Flash, Grok 4, Llama 3.3 70B, Claude Sonnet 4, DeepSeek-V2.5), and four RAG configurations, assessing clinical accuracy and patient-centered relevance. Validated by LLM-as-a-judge and board-certified ophthalmologists on internal/external datasets, EyeRAG outperformed Vanilla LLMs and standard RAG. LLM evaluations ranked EyeRAG highest (mean 1.61 ± 1.04 internal/1.72 ± 1.18 external). Experts assigned a mean rank of 1.0, with hallucinations falling to 3.3% (vs. 30% baseline). EyeRAG demonstrates the potential for safe, explainable, and clinically grounded language models to be developed as clinician-supervised aids designed specifically to bridge the communication gap between clinicians and patients. It serves as a tool for patient interpretation rather than clinical decision-making, assisting in patient counselling and tele-ophthalmology by translating complex findings into accessible dialog under professional oversight.