Related Experiment Video
Updated: Jun 27, 2026
06:45
Automated Joint Space Detection Improves Bone Segmentation Accuracy
Published on: November 28, 2025
Importance of the Quality of Annotation: Impact of Simulated Inter-Observer Variability on Deep Neural Network
Dominic LaBella1, Michaela Kop2, Xuan Qi3
1Department of Radiation Oncology, Duke University Medical Center, Durham, NC 27705, USA.
Bioengineering (Basel, Switzerland)
|June 26, 2026
Summary
Annotation variability in prostate segmentation models impacts performance. Models tolerate minor changes (1-5 mm) but degrade significantly with variations over 5 mm, highlighting annotation quality importance.
Area of Science:
- Radiology and Medical Imaging
- Artificial Intelligence in Medicine
- Biomedical Engineering
Background:
- Deep neural networks (DNNs) for prostate segmentation rely on manual annotations.
- The impact of annotation variability on DNN performance is not well understood.
Purpose of the Study:
- To investigate how annotation variability affects deep neural network performance for prostate segmentation.
- To determine the threshold of annotation variability that significantly impacts segmentation accuracy.
Main Methods:
- Prostate contours were manually delineated on 119 MR images.
- Synthetic radial modifications (1-10 mm) were applied to create varied training datasets.
- SegResNet models were trained and evaluated using Dice Similarity Coefficient (DSC).
Main Results:
- Mean test DSC decreased from 0.917 (baseline) to 0.856 at 10 mm modification.
- Models maintained DSC ≥ 0.90 with perturbations up to 5 mm.
- Performance declined significantly with annotation variability exceeding 5 mm.
Conclusions:
- Prostate segmentation models can tolerate modest annotation variability (≤ 5 mm).
- Substantial performance degradation occurs with annotation variability greater than 5 mm.
- Annotation quality is crucial for training and benchmarking DNN-based segmentation models.