Related Experiment Videos
Active Learning Under Expert-Budget Constraints: A Human-in-the-Loop Pipeline for Diabetic Retinopathy Lesion
Hyeok Kim1, Seok-Min Chang1, Bo-Young Lim2
1Department of Industrial Engineering, Seoul National University of Science and Technology, 232 Gongneung-ro, Nowon-gu, Seoul 01811, Republic of Korea.
Bioengineering (Basel, Switzerland)
|July 28, 2026
Summary
Optimizing active learning for Diabetic Retinopathy (DR) detection under expert time constraints is key. Staged active learning strategies, starting with random sampling and progressing to diversity-based methods, improve microaneurysm detection sensitivity with AI assistance.
Area of Science:
- Ophthalmology
- Medical Imaging
- Artificial Intelligence
Background:
- Diabetic Retinopathy (DR) diagnosis requires expert annotation, a bottleneck due to limited ophthalmologist availability.
- Active learning (AL) aims to reduce annotation costs but is constrained by expert time in clinical settings.
- Developing efficient AL strategies is crucial for clinical-grade DR detection models.
Purpose of the Study:
- To evaluate the effectiveness of different AL strategies under tight expert budget constraints for DR detection.
- To propose and assess the 'Virtuous Cycle' Human-in-the-Loop (HITL) pipeline for DR lesion annotation.
- To compare AI-assisted annotation with manual annotation in terms of time and detection accuracy.
Main Methods:
- Implemented a YOLOv8x object detector for DR lesions (microaneurysms, hemorrhages, exudates).
- Integrated four AL sampling strategies: Average Confidence, Random, Hybrid-Diversity, and Monte Carlo Dropout.
- Utilized an in-hospital annotation platform (Diavision Studio) for clinician refinement of AI pre-labels.
- Evaluated the pipeline on a real-world fundus dataset over eight AL rounds with limited expert time.
Main Results:
- Random sampling was effective initially, improving mean Average Precision (mAP@50) from 0.14 to 0.25.
- Hybrid-Diversity sampling achieved the highest mAP@50 (0.40), Precision (0.55), and Recall (0.41) by AL round 7.
- AI-assisted annotation showed no significant difference in labeling time but a significant increase in confirmed lesion detections (p=0.0058), especially for microaneurysms.
Conclusions:
- The choice of AL strategy should be staged: random sampling for initial model training and uncertainty/diversity sampling for model maturation.
- AI assistance offers a measurable gain in microaneurysm detection sensitivity with a manageable time cost.
- The Virtuous Cycle HITL pipeline effectively optimizes DR annotation under practical clinical expert-time limitations.