Deep-learning segmentation to guide bladder neck recognition in robot-assisted radical prostatectomy

To develop a deep-learning segmentation model for bladder neck dissection during robot-assisted radical prostatectomy (RARP) and evaluate its effectiveness in improving anatomical recognition in surgical videos across surgeons with varying levels of experience.

From April 2022 to March 2023, we collected 25 RARP procedures at Fujita Health University, specifically extracting bladder neck dissection segments. Three architectures (U-Net, U-Net++, and DeepLabv3+) were trained on 979 annotated frames from 10 RARP procedures performed by a single expert surgeon. The best-performing model was adopted as the base model for two-step, entropy-based active learning. Step one included selecting informative frames from the same videos used for the base model, with finer temporal sampling, to construct the prototype model. In step two, diverse procedures from additional surgeons were used to build the final model. Segmentation performance was evaluated using Intersection over Union (IoU; mean IoU [mIoU]). Clinical utility was assessed by quantifying the recognition of anatomical structures with and without guidance from the segmentation model among seven urologists (two expert surgeons and five novice surgeons). Anatomical recognition was evaluated by comparing annotation time and the average IoU of the bladder and prostate (bpIoU).

DeepLabv3+ with LogCosh loss achieved the highest performance (mIoU = 0.815, 95% confidence interval [CI] 0.699-0.931) and was adopted as the base model. Active learning improved generalisability, increasing mIoU on the novice evaluation set from 0.709 (95% CI 0.584-0.833) to 0.713 (95% CI 0.586-0.839). Regarding the assessment of clinical utility, among novice surgeons, model guidance significantly reduced annotation time by 28 s and improved annotation quality of bpIoU by 0.124. No significant changes were observed among expert surgeons.

The developed segmentation model enhanced the speed and performance of anatomical recognition, particularly for novice surgeons, and may support surgical education.