A deep learning approach based on YOLO v11 for automatic detection of jaw cysts

Jaw cysts are frequent radiolucent lesions in dentistry that can present diagnostic difficulties due to their similar radiographic appearance. This study aimed to develop an AI-based detection and classification system for jaw cysts using the YOLO v11 deep learning model on panoramic radiographs.

A total of 311 panoramic images (211 cystic, 100 normal) were labeled and augmented. The YOLO v11 Small model was trained in both multi-class (distinguishing between dentigerous cysts [DCs], odontogenic keratocysts [OKCs], and radicular cysts [RCs]) and single-class configurations (detecting cysts without type differentiation). Performance metrics included precision, recall, F1 score, and mean average precision (mAP).

In the multi-class model, the system achieved an mAP of 86%, with precision of 84%, recall of 82%, and F1 score of 83%. Class-wise mean accuracies were 91% for DCs, 85% for OKCs, and 82% for RCs. The single-class model showed slightly lower performance with an mAP of 84% and F1 score of 81%.

YOLO v11 demonstrated high accuracy in detecting jaw cysts, indicating its potential to support dental diagnostics. Further validation on larger and balanced datasets is recommended to enhance generalizability.