Learning how to robustly estimate camera pose in endoscopic videos.

Area of Science:

• Computer Vision
• Medical Robotics
• Surgical Technology

Background:

• Accurate endoscope pose tracking is crucial for computer-assisted interventions in minimally invasive surgery.
• Challenges include varying illumination, tissue deformation, and organ motion, hindering precise camera localization.
• Existing methods struggle with the dynamic and complex nature of surgical environments.

Purpose of the Study:

• To develop a robust solution for stereo endoscope camera pose estimation in challenging surgical scenarios.
• To improve surgical scene understanding for advanced intervention-assisting systems.
• To address limitations of current pose estimation techniques in dynamic surgical environments.

Main Methods:

• Proposes a stereo endoscope solution estimating depth and optical flow to minimize geometric losses for camera pose.
• Introduces learned adaptive per-pixel weight mappings to balance image content contributions.
• Employs a Deep Declarative Network combining deep learning expressiveness with geometric optimization for robustness.

Main Results:

• The method outperforms state-of-the-art approaches, especially in scenarios with tissue deformation and organ motion.
• Adaptive weight mappings effectively reduce the influence of ambiguous image regions, such as deforming tissues.
• Validation on public (SCARED) and new in vivo (StereoMIS) datasets demonstrates effectiveness across diverse surgical settings.

Conclusions:

• The developed solution robustly estimates camera pose in difficult endoscopic surgical scenes.
• Contributions enhance tasks like simultaneous localization and mapping (SLAM) and 3D reconstruction.
• Advances surgical scene understanding, paving the way for improved minimally invasive surgery technologies.