Low-Rank Fine-Tuning Meets Cross-modal Analysis: A Robust Framework for Age-Related Macular Degeneration...

Area of Science:

• Ophthalmology
• Medical Imaging
• Artificial Intelligence

Background:

• Age-related macular degeneration (AMD) is a leading cause of vision loss in the elderly.
• Current diagnostic methods using single imaging modalities like color fundus photography (CFP) and optical coherence tomography (OCT) have limitations in capturing complex AMD pathology.
• Accurate and early diagnosis of AMD is crucial for effective management and preventing irreversible vision impairment.

Purpose of the Study:

• To develop and validate an innovative multi-modal deep learning framework for enhanced age-related macular degeneration (AMD) categorization.
• To integrate features from CFP and OCT imaging for a more comprehensive AMD diagnosis.
• To improve the efficiency and performance of multi-modal AMD classification models.

Main Methods:

• A multi-modal deep learning framework utilizing vision transformer models for feature extraction from CFP and OCT images.
• Deep canonical correlation analysis (DCCA) for nonlinear feature mapping and fusion to maximize cross-modal correlations.
• Integration of the low-rank adaptation (LoRA) technique to reduce computational complexity and enhance parameter efficiency.

Main Results:

• The proposed framework achieved high performance metrics on the MMC-AMD dataset, including an F1-score of 0.948, AUC-ROC of 0.991, and accuracy of 0.949.
• The model significantly outperformed existing single-modal and multi-modal baseline approaches for AMD categorization.
• The LoRA technique enabled superior performance with only 0.49% of trainable parameters compared to full fine-tuning.

Conclusions:

• The developed multi-modal deep learning framework effectively integrates CFP and OCT data for superior AMD diagnosis.
• The use of DCCA and LoRA offers an efficient and high-performing solution for complex pathological categorization in AMD.
• This approach holds significant promise for improving the accuracy and efficiency of AMD detection in clinical settings.