SAGE: SLAM with Appearance and Geometry Prior for Endoscopy

Area of Science:

• Medical Imaging
• Computer Vision
• Robotics

Background:

• Endoscopic applications like surgical navigation require real-time endoscope tracking and 3D anatomical reconstruction.
• Existing methods struggle with monocular endoscopic video due to challenges like texture scarcity and illumination variations.

Purpose of the Study:

• To develop a robust Simultaneous Localization and Mapping (SLAM) system for real-time endoscope tracking and 3D reconstruction from monocular endoscopic video.

Main Methods:

• Developed a SLAM system integrating learning-based appearance and geometry priors with factor graph optimization.
• Employed an end-to-end differentiable training pipeline for learning pair-wise image alignment.
• Utilized explicit learning of appearance and geometry priors for core SLAM components.

Main Results:

• The proposed SLAM system demonstrates robust performance in challenging endoscopic conditions, including texture scarcity and illumination variations.
• The system shows good generalization to unseen endoscopes and subjects.
• Achieved favorable performance compared to state-of-the-art feature-based SLAM systems.

Conclusions:

• The developed SLAM system effectively addresses key challenges in endoscopic real-time 3D reconstruction and localization.
• The approach offers a promising solution for enhancing endoscopic navigation and surgical guidance.