Robotically assisted augmented reality system for identification of targeted lymph nodes in laparoscopic...

To prove feasibility of multimodal and temporal fusion of laparoscopic images with preoperative computed tomography scans for a real-time in vivo-targeted lymph node (TLN) detection during minimally invasive pelvic lymphadenectomy and to validate and enable such guidance for safe and accurate sentinel lymph node dissection, including anatomical landmarks in an experimental model.

A measurement campaign determined the most accurate tracking system (UR5-Cobot versus NDI Polaris). The subsequent interventions on two pigs consisted of an identification of artificial TLN and anatomical landmarks without and with augmented reality (AR) assistance. The AR overlay on target structures was quantitatively evaluated. The clinical relevance of our system was assessed via a questionnaire completed by experienced and trainee surgeons.

An AR-based robotic assistance system that performed real-time multimodal and temporal fusion of laparoscopic images with preoperative medical images was developed and tested. It enabled the detection of TLN and their surrounding anatomical structures during pelvic lymphadenectomy. Accuracy of the CT overlay was > 90%, with overflow rates < 6%. When comparing AR to direct vision, we found that scores were significatively higher in AR for all target structures. AR aided both experienced surgeons and trainees, whether it was for TLN, ureter, or vessel identification.

This computer-assisted system was reliable, safe, and accurate, and the present achievements represent a first step toward a clinical study.