Convolutional Network With Twofold Feature Augmentation for Diabetic Retinopathy Recognition From Multi-Modal Images

Area of Science:

• Ophthalmology
• Medical Imaging
• Artificial Intelligence

Background:

• Diabetic Retinopathy (DR) diagnosis often requires large labeled datasets.
• Multi-modal retinal imaging is increasingly available for diabetes patients.

Purpose of the Study:

• To develop a deep learning approach for Diabetic Retinopathy (DR) severity recognition using fundus images and wide-field swept-source optical coherence tomography angiography (SS-OCTA).
• To address the challenge of limited labeled datasets in medical image analysis.

Main Methods:

• Proposed a novel deep learning architecture, TFA-Net, featuring a backbone convolutional network and a Twofold Feature Augmentation (TFA) mechanism.
• The TFA mechanism utilizes weight-sharing convolution kernels and a Reverse Cross-Attention (RCA) stream for feature extraction and fusion.
• The model integrates information from fundus photography and SS-OCTA.

Main Results:

• Achieved a Quadratic Weighted Kappa rate of 90.2% on the KHUMC dataset.
• The RCA stream demonstrated robustness, achieving 94.8% mean Accuracy and 99.4% Area Under the ROC Curve on the Messidor dataset, outperforming state-of-the-art methods.
• The model effectively leverages multi-modal data for DR biomarker identification.

Conclusions:

• Deep learning models utilizing feature-space regularization for multi-modal data amalgamation are effective.
• The proposed TFA-Net approach reduces reliance on extensive labeled data for DR severity recognition.
• The method enhances generalization ability, particularly in scenarios with small labeled datasets.