Comparative evaluation of YOLO models for retained root fragment detection

This study aimed to develop and comparatively evaluate four YOLO nano-scale one-stage object detectors-YOLOv8n, YOLOv10n, YOLO11n, and YOLO12n-for the automated detection of retained root fragments in dental radiographic images, under identical preprocessing, training, and evaluation protocols.

A curated dataset was organized in the YOLO format and evaluated using a nested 5-fold cross-validation strategy to prevent data leakage. An ablation study was additionally conducted to validate the combined bilateral filtering and histogram equalization (BF + HE) preprocessing pipeline. Four YOLO models were fine-tuned from official pretrained weights over 200 epochs. Detection performance was assessed at both the epoch level and the decision-threshold level, reporting the unbiased mean and standard deviation across the independent test folds for precision, sensitivity, F1 score, mean average precision (mAP), and false positives per image (FP/image).

In the primary multi-model benchmark, YOLO12n achieved the highest overall detection performance across the held-out test folds, yielding a mean best-epoch F1 score of 0.885 ± 0.007, mAP@0.5 of 0.919 ± 0.021, and mAP@0.5:0.95 of 0.447 ± 0.025 (reaching an optimal-threshold F1 score of 0.888 ± 0.016 with 0.346 ± 0.052 FP/image). YOLO11n followed closely as the second-best performing architecture (F1 = 0.874 ± 0.012, mAP@0.5:0.95 = 0.445 ± 0.019). Furthermore, the preprocessing ablation study confirmed that the hybrid BF + HE pipeline improved detection performance over raw radiographs, elevating YOLO12n's F1 score from 0.856 ± 0.013 to 0.888 ± 0.016 and mAP@0.5:0.95 from 0.438 ± 0.011 to 0.461 ± 0.008. All models demonstrated stable convergence behavior.

Modern YOLO architectures, particularly YOLO12n and YOLO11n, demonstrate technical efficacy for automated RRF localization. The observed performance advantages of YOLO12n are associated with its overall architectural refinements, though dedicated modular ablation studies are warranted to isolate the exact contribution of individual attention mechanisms. These findings demonstrate the technical feasibility of lightweight one-stage object detectors for identifying retained root fragments in dental panoramic radiographs, serving as a baseline for future prospective clinical evaluations.