Clean visual field reconstruction in robot-assisted laparoscopic surgery based on dynamic prediction

Area of Science:

• Medical Robotics
• Computer-Aided Surgery
• Surgical Visualization

Background:

• Robot-assisted minimally invasive surgery (RAMIS) is crucial for complex operations.
• Surgical instruments can obscure critical anatomical details in narrow surgical fields.
• Accurate visualization of lesions and adjacent structures is vital for surgical precision.

Purpose of the Study:

• To develop a surgical scene reconstruction method for RAMIS.
• To improve surgeon's judgment by removing instrument obstruction and predicting obscured regions.
• To enhance the visualization of lesions and adjacent vessels/lacunae.

Main Methods:

• Instrument tracking and segmentation using a modified U-Net architecture (ResNet101 encoder, custom decoder).
• Extension of instrument shaft segmentation boundaries via image filtering and real-time index mask algorithm.
• Soft tissue deformation prediction using dense optical flow, gravitational field, and entropy increase, integrated with morphological operations.

Main Results:

• The proposed method successfully tracks and segments surgical instruments.
• It accurately predicts obscured regions and reconstructs the course of important vessels/lacunae.
• Preliminary experiments and pre-clinical evaluations demonstrate effective performance.

Conclusions:

• The method provides a clean and comprehensive surgical scene for surgeons.
• It aids in reconstructing critical anatomical pathways, reducing the risk of inadvertent injuries.
• This technique has the potential to improve safety and outcomes in complex robotic surgeries.