Study on automatic recognition of stone composition in intraoperative endoscopic images-a single center study

Urinary stone composition critically influences treatment selection and recurrence prevention, yet current intraoperative assessment remains imprecise. This study aims to achieve intraoperative prediction of stone composition by applying a deep convolutional neural network (CNN) to routinely captured endoscopic images.

We retrospectively studied endoscopic images from stone-breaking surgeries in Beijing Tsinghua Changgung Hospital during 2022-12-2024-12. Images were captured before and after laser lithotripsy. Based on postoperative infrared spectroscopy, stones were divided into five categories. In total, 1780 images (1167 from RIRS, 613 from PCNL) were included and split into training and testing sets at an 8:2 ratio. Using ResNet-50 as the base model, only endoscopic digital images and stone classification data were input for minimal-supervision learning. After training, the model accuracy for each stone category surpassed 95%. The model was then tested on 20% of RIRS and RIRS+PCNL images, with 3D PCA and Grad-cam for visual analysis.

For the RIRS image test set: The precision was 93.8% for the calcium oxalate group (n = 147), 96.3% for the calcium oxalate mixed with a uric acid group (n = 30), 91.8% for the calcium oxalate mixed with carbonate apatite group (n = 106), 88.9% for the struvite mixed with calcium oxalate and carbonate apatite group (n = 16), and 100% for the stone free control group (n = 26) (Table 1). Total accuracy for CNN modeling is:94.16%, AUC:0.99, weighted F1-Score: 0.9353, weighted F1-score 95% CI: (0.9089, 0.9599), weighted Kappa: 0.9122, weighted Kappa 95% CI: (0.8523, 0.9569).For the RIRS+PCNL image test set: The precision was 94.9% for the calcium oxalate group (n = 195), 98.7% for the calcium oxalate mixed with a uric acid group (n = 80), 92.2% for the calcium oxalate mixed with carbonate apatite group (n = 111), 100% for the struvite mixed with calcium oxalate and carbonate apatite group (n = 29), and 93.8% for the stone free control group (n = 26) (Table 2).Total accuracy: 95.92%, AUC: 0.99, weighted F1-Score: 0.9508, weighted F1-score 95% CI: (0.9304, 0.9708) weighted Kappa: 0.9357, weighted Kappa 95% CI: (0.8907, 0.9678).The 3D PCA projection results are as follows: PC1: 0.2723 (27.23%), PC2: 0.1102 (11.02%), PC3: 0.0801 (8.01%), Cumulative: 46.26%.

This study shows deep CNNs can identify renal stone compositions from intraoperative endoscopic images, differentiating pure and mixed components. This analysis is an alternative to traditional methods and has the potential to improve treatment effectiveness.