AI-powered semantic segmentation model for enhanced ureteral mapping and real-time instrument feedback in robotic...

Ureteral injury is a serious complication of colorectal surgery. Current intraoperative localization methods, including indocyanine green (ICG) fluorescence and dissection with direct visualization can be time-consuming, invasive, and technically limited. We developed a deep learning model which provides real-time instrument-proximity warnings and ureter identification without ICG during robotic colorectal surgery. Deidentified videos of robotic-assisted low anterior resections and sigmoid colectomies were annotated for ureter and surgical instrument locations across multiple operative phases and camera perspectives. A convolutional neural network was trained for semantic ureter segmentation and instrument detection. Model performance was evaluated using mean average precision (mAP) and F1 score. The system was also configured to generate visual warnings when instruments approached the predicted ureter location. The model demonstrated high detection accuracy, achieving a mAP of 0.8821 which is considerably better than most real-world models (mAP range: 0.5-0.9). The model also showed a high F1 score of 0.8347 demonstrating balance between precision and recall. Performance remained robust across variable anatomy, operative fields, and viewing angles. The model consistently localized the ureters and surgical instruments in real-time for previously unseen video frames while also generating proximity warnings when dissecting near the ureter. This AI-powered semantic segmentation platform enables real-time, contrast-free ureter mapping and instrument-proximity warnings during robotic colorectal surgery. This model provides an intraoperative safeguard for prevention of ureteral injury to improve surgical outcomes. Further work will focus on external validation, prospective evaluation, and integration with robotic platforms to support real-time surgical decision-making across diverse procedures, surgeons, patients, and institutional settings.