Multi-modality multi-label ocular abnormalities detection with transformer-based semantic dictionary learning.

Area of Science:

• Ophthalmology
• Artificial Intelligence
• Medical Imaging

Background:

• Early detection of ocular abnormalities is crucial for preventing blindness.
• Computer-aided diagnosis (CAD) systems analyze retinal images like Color Fundus Photography (CFP) for abnormality detection.
• Detecting rare ocular abnormalities remains challenging due to limited features.

Purpose of the Study:

• To propose a multi-label detection model for 28 ocular abnormalities using transformer-based semantic dictionary learning on CFP.
• To address the challenge of detecting rare abnormalities by integrating linguistic features.
• To enhance the performance of ocular abnormality detection by emphasizing the semantic dictionary's role.

Main Methods:

• Developed a transformer-based semantic dictionary learning model for multi-label ocular abnormality detection.
• Incorporated a co-occurrence dependency factor from label linguistic features to improve rare label detection.
• Treated the semantic dictionary as a primary component, acting as a query for spatial features (key/value).

Main Results:

• The proposed method achieved top performance within the top 30% on the RFMiD dataset challenge evaluation set.
• Demonstrated that prioritizing the semantic dictionary significantly increases detection performance.
• Outperformed methods where the semantic dictionary was considered a weaker factor.

Conclusions:

• Transformer-based semantic dictionary learning offers a robust approach for multi-label ocular abnormality detection from CFP.
• Integrating linguistic features effectively addresses the challenge of detecting rare ocular conditions.
• The model's architecture, emphasizing the semantic dictionary, significantly boosts diagnostic accuracy in ophthalmology.