Related Experiment Video
Updated: Jun 27, 2026
Automated Joint Space Detection Improves Bone Segmentation Accuracy
Published on: November 28, 2025
Importance of the Quality of Annotation: Impact of Simulated Inter-Observer Variability on Deep Neural Network
Dominic LaBella1, Michaela Kop2, Xuan Qi3
1Department of Radiation Oncology, Duke University Medical Center, Durham, NC 27705, USA.
Background:
Deep neural network based prostate segmentation depends on manual annotations, yet the effect of annotation variability on model performance remains underexplored.
Methods:
Prostate contours were manually delineated by an expert clinician on 119 T2-weighted MR images from the PROSTATEx Challenge 2017 training dataset, and slice-wise synthetic radial modifications of 1-10 mm were applied to create 10 modified training datasets plus an unmodified baseline. Identical SegResNet models were trained with Auto3DSeg/MONAI and evaluated against unmodified validation and test sets using the Dice similarity coefficient (DSC).
Results:
Mean test DSC decreased from 0.917 for the baseline model to 0.856 at 10 mm modification. Models trained with small annotation perturbations of 1-5 mm maintained DSC values of at least 0.90, whereas performance declined significantly beyond 5 mm. Pairwise DSC agreement across modified annotations also fell as modification amplitude increased.
Conclusions:
Prostate segmentation models tolerated modest annotation variability but degraded substantially when variability exceeded 5 mm, underscoring the importance of annotation quality when training and benchmarking DNN-based automated segmentation models.