Advancing Apex Detection: a Comparative Study of Deep Learning Models on Panoramic Radiographs

The open or closed root apex identification is one of the most important factors determining endodontic treatment options in pediatric dentistry. This study presents a deep learning-based automatic apical developmental status detection system on mandibular permanent first and second molar teeth, aiming to reduce the subjective evaluation and time constraints brought by manual diagnosis. A total of 1498 anonymous panoramic radiographs of pediatric patients between the ages of 5 and 14 were used as a dataset for experiments. The performances of one and two-stage deep learning techniques, mainly YOLO and Faster R-CNN, were compared in detecting open and closed apexes in panoramic radiographs. The dataset was first divided into a model-development set and a fixed independent test set. A fivefold cross-validation strategy was then applied only to the model-development set. For each fold, the best model was selected based on validation performance and subsequently evaluated on the same independent test set. Final performance was reported as the mean independent test-set performance across the fivefold-trained models using mean average precision (mAP), recall, and precision. In addition, explainable artificial intelligence analysis was performed to highlight the image regions contributing to the models' decision-making processes, and a descriptive error analysis was conducted to identify common factors associated with misclassifications. YOLOv8x achieved the highest independent test-set mAP@0.5 value of 0.956, while Faster R-CNN with a ResNet-101 backbone showed a competitive but lower value of 0.908. Additionally, YOLOv8x demonstrated high performance in distinguishing between open and closed apexes with an average precision of 0.904 and a recall of 0.942 on the test set. YOLOv8x showed the best overall performance, and the explainability findings indicated that correct detections were generally associated with focused attention on the apex region. The results suggest that deep learning-based object detection can provide promising support for apical developmental status assessment on panoramic radiographs. The proposed approach shows promise as a clinical decision-support tool, although further validation is required before routine implementation.