An integrated language-vision foundation model for conversational diagnostics and triaging in primary eye care

Area of Science:

• Artificial Intelligence in Healthcare
• Ophthalmology
• Medical Diagnostics

Background:

• Primary eye care faces challenges with a shortage of skilled professionals.
• Accurate and timely diagnosis of ocular diseases is crucial for patient outcomes.
• Existing AI models may lack the integrated capabilities for comprehensive eye care diagnostics.

Purpose of the Study:

• To introduce Meta-EyeFM, an integrated language-vision foundation model for conversational diagnostics and triaging in primary eye care.
• To evaluate the diagnostic performance of Meta-EyeFM in detecting ocular diseases, differentiating severity, identifying signs, and predicting systemic conditions.
• To compare Meta-EyeFM's accuracy against existing LLMs and human graders.

Main Methods:

• Development of Meta-EyeFM by integrating a large language model (LLM) with eight task-specific vision foundation models (VFMs).
• Dynamic routing of user queries and fundus photographs to the most appropriate VFMs.
• Evaluation of performance using metrics such as area under the receiver operating curve (AUC) for disease detection, severity differentiation, sign identification, and systemic condition prediction.

Main Results:

• Meta-EyeFM achieved high accuracy in detecting ocular diseases (AUC ≥91.2%), differentiating severity (AUC ≥82%), identifying signs (AUC ≥77.9%), and predicting diabetes (AUC ≥79.8%).
• The model demonstrated superior accuracy (11%-43% higher) compared to Gemini-1.5-flash and GPT-4o LLMs.
• Meta-EyeFM generally outperformed junior ophthalmologists and optometrists in detecting various eye diseases.

Conclusions:

• Meta-EyeFM serves as a valuable diagnostic decision support tool in community eye care settings.
• Its conversational interface and generalizability enhance its utility for vision screening and disease triage.
• The AI model offers a scalable and explainable solution to address the global scarcity of eye care expertise.