Geometric Correspondence-Based Multimodal Learning for Ophthalmic Image Analysis.

Area of Science:

• Ophthalmology
• Medical Imaging
• Artificial Intelligence

Background:

• Color fundus photography (CFP) and Optical coherence tomography (OCT) are key for diagnosing retinal diseases.
• Current automated methods struggle to effectively use complementary data from multiple imaging modalities.

Purpose of the Study:

• To develop a novel multimodal learning method for enhanced automated diagnosis of retinal diseases.
• To effectively leverage correlated information from CFP and OCT images.

Main Methods:

• Proposed GeCoM-Net (geometric correspondence-based multimodal learning network) for fusing CFP and OCT images.
• Incorporated geometric correspondence between OCT slices and CFP regions for feature learning.
• Developed a feature selection strategy for discriminative OCT representations.

Main Results:

• GeCoM-Net achieved superior performance in diagnosing diabetic macular edema (DME), impaired visual acuity (VA), and glaucoma.
• Demonstrated improved AUROC scores of 0.4% for DME, 1.9% for VA, and 2.9% for glaucoma compared to state-of-the-art methods.

Conclusions:

• GeCoM-Net effectively fuses CFP and OCT data by explicitly modeling geometric relationships.
• The method offers a significant advancement in automated multimodal retinal disease diagnosis.