Glaucoma Detection and Structured OCT Report Generation via a Fine-tuned Multimodal Large Language Model

Area of Science:

• Ophthalmology and Artificial Intelligence
• Medical Imaging Analysis
• Clinical Decision Support Systems

Background:

• Optic nerve head (ONH) Optical Coherence Tomography (OCT) scans are crucial for diagnosing glaucoma.
• Automated analysis of OCT scans can improve efficiency and accuracy in glaucoma detection and monitoring.
• Developing explainable AI models is essential for clinical adoption in medical diagnostics.

Purpose of the Study:

• To develop an explainable multimodal large language model (MM-LLM) for analyzing ONH OCT scans.
• To enable the MM-LLM to screen OCT scans for quality and detect glaucoma.
• To generate structured clinical reports detailing glaucoma diagnosis and sector-wise retinal nerve fiber layer (RNFL) thinning.

Main Methods:

• A retrospective cohort study utilized longitudinal data from the Diagnostic Innovations in Glaucoma Study (DIGS) and African Descent and Glaucoma Evaluation Study (ADAGES).
• A Llama 3.2 Vision-Instruct based MM-LLM was fine-tuned on 43,849 Spectralis ONH OCT scans.
• The model was evaluated on quality assessment, glaucoma detection, and RNFL thinning classification across seven sectors using standard metrics.

Main Results:

• The MM-LLM achieved 0.90 accuracy for quality triage and 0.86 accuracy for glaucoma detection.
• RNFL thinning prediction accuracy ranged from 0.83 to 0.94, with superior performance in global and temporal sectors.
• Text generation quality scores demonstrated strong alignment with reference clinical reports (e.g., ROUGE-1: 0.94 ± 0.08).

Conclusions:

• The fine-tuned MM-LLM accurately analyzes ONH OCT imaging, identifying quality issues and detecting glaucoma.
• The model provides valuable sectoral RNFL thinning assessments, supporting clinical OCT evaluation.
• This AI approach shows promise as a scalable clinical decision support tool, warranting further validation.