Uncertainty-inspired open set learning for retinal anomaly identification

Area of Science:

• Ophthalmology
• Artificial Intelligence
• Medical Image Analysis

Background:

• Artificial intelligence (AI) models struggle to identify novel or unseen retinal conditions.
• Accurate classification of retinal anomalies is crucial for early diagnosis and treatment.
• Current AI systems lack robust mechanisms for handling out-of-distribution samples in retinal imaging.

Purpose of the Study:

• To develop an uncertainty-inspired open set (UIOS) model for improved recognition and classification of retinal anomalies.
• To enhance the reliability of AI in real-world clinical settings by addressing the challenge of unseen classes.
• To provide a confidence measure alongside classification predictions for retinal fundus images.

Main Methods:

• Developed an uncertainty-inspired open set (UIOS) model trained on fundus images of 9 retinal conditions.
• Incorporated an uncertainty score calculation alongside category probability assessment.
• Implemented a thresholding strategy to evaluate the model's performance on diverse datasets.

Main Results:

• The UIOS model achieved significantly higher F1 scores (99.55% internal, 97.01% external TC, 91.91% unseen TC) compared to a standard AI model (92.20%, 80.69%, 64.74%).
• UIOS correctly predicted high uncertainty scores for non-target categories, including retinal diseases, low-quality images, and non-fundus images.
• The model demonstrated robust performance in distinguishing between known and unknown retinal conditions.

Conclusions:

• The UIOS model offers a robust solution for real-world screening of retinal anomalies by effectively handling unseen classes.
• The uncertainty score provides a valuable tool for identifying cases requiring manual expert review, improving diagnostic safety.
• This approach enhances the practical applicability of AI in ophthalmology for anomaly detection.