How much data do you need? An analysis of pelvic multi-organ segmentation in a limited data context

Training deep learning models generally requires large, costly datasets which can limit their application towards in-house segmentation tasks. This study investigates the trade-off in dataset size within the context of pelvic multi-organ MR segmentation where we evaluate the performance of nnU-Net, a well-known segmentation model, under conditions of limited domain and data availability. 12 participants undergoing treatment on an Elekta Unity were recruited, acquiring 58 MR images, with 4 participants (12 images) withheld for testing. Prostate, seminal vesicles (SV), bladder and rectum were contoured in each image by a radiation oncologist. Seven models were trained on progressively smaller subsets of the training dataset, simulating a limited dataset setting. To investigate the efficacy of data augmentation, another set of identical models were trained without augmentation. The performance of the networks was evaluated via the Dice Similarity Coefficient, mean surface distance, and 95% Hausdorff distance metrics. When trained with entire training dataset (46 images), the model achieved a mean Dice coefficient of 0.903 (Prostate), 0.851 (SV), 0.884 (Rectum) and 0.967 (Bladder). Segmentation performance remained stable when the number of training sets was > 12 images from 4 participants, but rapidly dropped in smaller data subsets. Data augmentation was found to be influential across all dataset sizes, but especially in very small datasets. This study demonstrated nnU-Net's proficiency in performing male pelvic multi-organ segmentation under a limited domain, a single scanner, and under limited data constraints. We found that the performance degradation was often modest until a threshold is reached (12 images), below which it dropped significantly. Data augmentation improved performance across all data sizes, but especially for very small datasets. We conclude that nnU-Net's low data requirement can be advantageous for in-house cases with consistent protocol and scarce data availability.