Toward Multimodal Conversational AI for Age-Related Macular Degeneration.

Area of Science:

• Ophthalmology
• Artificial Intelligence
• Medical Imaging

Background:

• Deep learning models excel at retinal disease detection but lack clinical reasoning.
• Multimodal large language models (MLLMs) integrate diagnostic predictions with dialogue for clinical support.
• Existing systems offer static predictions, hindering interactive patient counseling and decision-making.

Purpose of the Study:

• To develop and evaluate OcularChat, an MLLM fine-tuned for diagnosing age-related macular degeneration (AMD) using color fundus photographs (CFPs).
• To assess OcularChat's ability to provide reasoned predictions, clinical explanations, and interactive dialogue for AMD classification.
• To compare OcularChat's performance against existing MLLMs and ophthalmologist evaluations.

Main Methods:

• Fine-tuning Qwen2.5-VL, an MLLM, using simulated patient-physician dialogues and CFPs.
• Training OcularChat on 705,850 simulated dialogues and 46,167 CFPs for AMD feature identification and prediction.
• Evaluating OcularChat's classification accuracy on AREDS and AREDS2 datasets and comparing its clinical grading scores with a baseline model and ophthalmologist assessments.

Main Results:

• OcularChat achieved high accuracies for AMD diagnostic tasks on AREDS (0.954 for advanced AMD, 0.849 for pigmentary abnormalities, 0.678 for drusen size), outperforming existing MLLMs.
• OcularChat demonstrated superior performance on all AREDS2 tasks.
• Ophthalmologist evaluations showed OcularChat received higher mean scores than a baseline model for AMD severity classification and overall impression.

Conclusions:

• OcularChat exhibits strong objective performance in AMD classification and provides accurate, interpretable diagnostic reasoning.
• The MLLM enables clinically relevant explanations and interactive dialogue, enhancing patient counseling and decision-making.
• MLLMs like OcularChat hold potential for accurate, interpretable, and clinically useful image-based diagnosis of retinal diseases such as AMD.