GlanceSeg: Real-Time Microaneurysm Lesion Segmentation With Gaze-Map-Guided Foundation Model for Early Detection of
Abstract:
Early-stage diabetic retinopathy (DR) presents challenges in clinical diagnosis due to inconspicuous and minute microaneurysms (MAs), resulting in limited research in this area. Additionally, the potential of emerging foundation models, such as the segment anything model (SAM), in medical scenarios remains rarely explored. In this work, we propose a human-in-the-loop, label-free early DR diagnosis framework called GlanceSeg, based on SAM. GlanceSeg enables real-time segmentation of MA lesions as ophthalmologists review fundus images. Our human-in-the-loop framework integrates the ophthalmologist's gaze maps, allowing for rough localization of minute lesions in fundus images. Subsequently, a saliency map is generated based on the located region of interest, which provides prompt points to assist the foundation model in efficiently segmenting MAs. Finally, a domain knowledge filtering (DKF) module refines the segmentation of minute lesions. We conducted experiments on two newly-built public datasets, i.e., IDRiD and Retinal-Lesions, and validated the feasibility and superiority of GlanceSeg through visualized illustrations and quantitative measures. Additionally, we demonstrated that GlanceSeg improves annotation efficiency for clinicians and further enhances segmentation performance through fine-tuning using annotations. The clinician-friendly GlanceSeg is able to segment small lesions in real-time, showing potential for clinical applications.
Insights
GlanceSeg, a novel framework, uses the segment anything model (SAM) for real-time, label-free diagnosis of early diabetic retinopathy (DR). It assists ophthalmologists in segmenting microaneurysms (MAs) for improved clinical detection.
Area of Science:
- Ophthalmology
- Medical Imaging
- Artificial Intelligence
Background:
- Early-stage diabetic retinopathy (DR) diagnosis is challenging due to subtle microaneurysms (MAs).
- Foundation models like the segment anything model (SAM) have underexplored potential in medical imaging.
- Accurate segmentation of minute lesions is crucial for timely DR detection.
Purpose of the Study:
- To introduce GlanceSeg, a human-in-the-loop, label-free framework for early DR diagnosis using SAM.
- To enable real-time segmentation of MA lesions during clinical review of fundus images.
- To leverage ophthalmologist gaze and domain knowledge for precise lesion identification.
Main Methods:
- Integrating ophthalmologist gaze maps for lesion localization.
- Generating saliency maps to guide SAM for MA segmentation.
- Employing a domain knowledge filtering (DKF) module for refining segmentation.
- Utilizing the segment anything model (SAM) with a human-in-the-loop approach.
Main Results:
- GlanceSeg demonstrated feasibility and superiority on IDRiD and Retinal-Lesions datasets.
- The framework enables real-time segmentation of small lesions.
- GlanceSeg improved annotation efficiency for clinicians.
- Fine-tuning enhanced segmentation performance.
Conclusions:
- GlanceSeg offers a promising, clinician-friendly tool for early diabetic retinopathy diagnosis.
- The human-in-the-loop framework effectively segments minute microaneurysms in real-time.
- This approach highlights the potential of foundation models in medical diagnostics.
