Agreement between artificial intelligence and periodontists in the detection of periodontal bone loss on panoramic radiographs

To evaluate the agreement between radiographic bone level measurements generated by a deep learning model and those derived from expert periodontist annotations on panoramic radiographs.

In this retrospective single-center radiographic study, periodontal bone loss was segmented by a YOLOv8x-based deep learning model and independently manually segmented by four periodontists. The artificial intelligence (AI)- and periodontist-generated segmentations were converted into point clouds for radiographic bone level measurements, and the overall agreement and that for six tooth regions was assessed using Bland-Altman analysis, with additional error and association metrics calculated.

Across 2,037 teeth, the mean absolute error (MAE) between the AI-derived and periodontist consensus measurements was 5.08% points of root length. The measurements revealed a strong association (Pearson's r = 0.80, Spearman's ρ = 0.80). Agreement was highest in the upper anterior region (MAE = 4.42; R² = 0.75; r = 0.90) and lowest in the lower molar region (MAE = 7.66; R² = 0.26; r = 0.65). Bland-Altman analysis revealed a mean bias of 1.61% points, with wide limits of agreement across all regions.

AI-derived measurements were strongly associated periodontist consensus measurements, with only a small systematic difference. However, the limits of agreement, particularly in the molar regions, were too wide to support the use of AI-derived measurements for individual tooth-level assessment without clinician oversight.

The evaluated deep learning system may facilitate radiographic bone level assessment on panoramic radiographs but currently requires clinician oversight, especially in the molar regions.