Related Experiment Video
Updated: May 31, 2026
07:35
In Vivo Vascular Injury Readouts in Mouse Retina to Promote Reproducibility
Published on: April 21, 2022
Retinal lesion annotation on fundus imaging: an interobserver variability study
Gabriel Lepetit-Aimon1, Clément Playout2,3, Marie Carole Boucher2,3
1Polytechnique Montréal, QC H3T 1J4, Montréal, Canada. gabriel.lepetit-aimon@polymtl.ca.
Scientific Reports
|May 28, 2026
Summary
Manual annotation of diabetic retinopathy lesions shows high variability among experts. A semi-automated workflow significantly improved agreement and accuracy in lesion segmentation for better DR diagnosis.
Area of Science:
- Ophthalmology and Medical Imaging
- Artificial Intelligence in Healthcare
Background:
- Accurate segmentation of retinal lesions in fundus images is crucial for diabetic retinopathy (DR) diagnosis.
- The reliability of manual annotation by clinical experts, essential for training machine learning models, has not been thoroughly evaluated.
Purpose of the Study:
- To quantify interobserver variability in manual pixel-wise annotation of key retinal lesions (microaneurysms, hemorrhages, exudates, cotton-wool spots).
- To evaluate the effectiveness of a semi-automated workflow in improving the consistency and accuracy of retinal lesion segmentation.
Main Methods:
- Three senior retinologists independently annotated 51 fundus images from the MAPLES-DR dataset for microaneurysms, hemorrhages, hard exudates, and cotton-wool spots.
- Interobserver variability was assessed using metrics such as lesion detection rates, coordinate differences, and mean Intersection over Union (IoU).
- A semi-automated workflow involving correction of model-generated pre-segmentations was evaluated.
Main Results:
- Manual segmentation exhibited substantial variability, with 38% of lesions missed by other observers and a mean interobserver IoU of 0.47.
- Despite high variability in lesion annotation, inter-expert DR grading assessments were largely concordant.
- The semi-automated workflow significantly improved agreement, achieving a mean IoU of 0.71 and preserving over 90% of lesions from pre-segmentations.
Conclusions:
- Manual annotation of retinal lesions by experts is inconsistent, highlighting challenges in developing reliable AI models based solely on manual segmentation.
- A semi-automated approach, where experts refine AI-generated segmentations, substantially enhances interobserver agreement and accuracy for retinal lesion detection.
- Semi-automated workflows show promise for improving the efficiency and reliability of data annotation in diabetic retinopathy research.
