Cross-modal attention network for retinal disease classification based on multi-modal images.

Area of Science:

• Ophthalmology
• Medical Imaging
• Artificial Intelligence

Background:

• Multi-modal eye disease screening enhances diagnostic accuracy by integrating information from diverse sources.
• Current automated multi-modal diagnosis methods often overlook spatial correlations between images, focusing instead on individual modality specificity.

Purpose of the Study:

• To develop a novel cross-modal retinal disease diagnosis network (CRD-Net) for improved multi-modal retinal image analysis.
• To effectively extract and integrate relevant features from different imaging modalities for enhanced disease diagnosis.

Main Methods:

• Introduction of a cross-modal attention (CMA) module to adaptively focus on salient lesion features across modalities.
• Development of multiple loss functions to fuse features, considering modality correlation for training.
• Implementation of a multi-modal retinal image classification network.

Main Results:

• The proposed CRD-Net demonstrated superior performance compared to existing single-modal and multi-modal methods.
• Experimental validation on three public datasets confirmed the effectiveness of the CRD-Net approach.
• The network successfully leveraged spatial correlations and cross-modal features for accurate diagnosis.

Conclusions:

• CRD-Net offers a significant advancement in automated multi-modal retinal disease diagnosis.
• The integration of cross-modal attention and feature fusion strategies improves diagnostic accuracy.
• This approach holds promise for more precise and reliable screening of eye conditions.