SeeSaw: Learning Soft Tissue Deformation From Laparoscopy Videos With GNNs

Area of Science:

• Medical Imaging
• Computer Vision
• Surgical Robotics

Background:

• Laparoscopic surgery faces challenges with soft tissue deformation and temporary loss of view, hindering image-guided navigation.
• Existing methods for registration and localization are unreliable with dynamic, shape-changing soft tissues.
• Revisiting previously seen areas is problematic in dynamic surgical contexts, limiting technology adoption.

Purpose of the Study:

• To develop a novel method for estimating deformed soft tissue states from observable regions during laparoscopic surgery.
• To address the limitations of current image-guided surgery techniques in dynamic environments.
• To improve the reliability of surgical navigation by compensating for non-rigid tissue motion.

Main Methods:

• A Graph Neural Network (GNN) was employed to learn and estimate soft tissue surface deformations.
• Training data was generated from semi-automatically processed stereo laparoscopic videos.
• Surface meshes were created using feature detection, depth estimation, and Delaunay triangulation.

Main Results:

• The GNN method successfully predicted displacements of previously visible soft tissue connected to currently visible regions.
• The approach demonstrated effectiveness on both patient and porcine surgical data.
• The method compensates for non-rigidity in abdominal endoscopic scenes using stereo laparoscopic videos.

Conclusions:

• The novel GNN-based approach effectively estimates soft tissue deformation in dynamic laparoscopic surgery.
• This method enhances surgical navigation by compensating for tissue non-rigidity.
• The technique has potential applications in various dynamic surgical environments.