Multimodality Fusion Strategies in Eye Disease Diagnosis

Area of Science:

• Ophthalmology
• Medical Imaging
• Artificial Intelligence

Background:

• Diabetic eye diseases pose a significant risk of vision loss and blindness.
• Diagnosing these conditions is challenging with single imaging modalities.
• Multimodality fusion offers a promising approach to enhance diagnostic accuracy.

Purpose of the Study:

• To evaluate the effectiveness of three multimodality fusion strategies (early, joint, and late) for automated eye disease detection.
• To compare the performance of different convolutional neural network models within these fusion strategies.
• To identify the optimal fusion strategy for diagnosing eye diseases using multimodal retinal imaging.

Main Methods:

• Utilized three datasets: fundus fluorescein angiography, macula imaging, and combined digital retinal images.
• Employed state-of-the-art convolutional neural network models, including DenseNet121, InceptionResNetV2, and ResNet50V2.
• Implemented and compared early, joint, and late fusion strategies for integrating multimodal data.

Main Results:

• Early fusion (Type 0) with DenseNet121 achieved 99.45% average accuracy.
• Joint fusion with InceptionResNetV2 reached 99.58% average accuracy.
• Late fusion with ResNet50V2 attained a perfect 100% accuracy across all metrics, outperforming other methods and matching state-of-the-art models.

Conclusions:

• Late fusion is the optimal strategy for eye disease diagnosis compared to early and joint fusion.
• Multimodal data integration via late fusion significantly enhances diagnostic performance.
• The findings highlight the potential of late fusion with ResNet50V2 for accurate automated eye disease detection.