EndoStreamDepth: Temporally Consistent Monocular Depth Estimation for Endoscopic Video Streams

Area of Science:

• Medical Imaging
• Computer Vision
• Robotics

Background:

• Accurate depth perception is crucial for minimally invasive surgery.
• Existing monocular depth estimation methods struggle with the unique challenges of endoscopic video, such as limited field-of-view and variable lighting.
• Real-time, precise depth maps are needed for advanced applications like robotic surgery automation.

Purpose of the Study:

• To introduce EndoStreamDepth, a novel monocular depth estimation framework specifically designed for endoscopic video streams.
• To achieve accurate depth maps with sharp anatomical boundaries and temporal consistency.
• To enable real-time depth estimation for supporting downstream tasks in surgical automation.

Main Methods:

• Developed a framework processing individual frames with a temporal module for inter-frame information propagation.
• Integrated a single-frame depth network with endoscopy-specific transformations.
• Utilized multi-level Mamba temporal modules for enhanced accuracy and prediction stability.
• Implemented a hierarchical design with multi-scale supervision using complementary loss terms.

Main Results:

• EndoStreamDepth significantly outperforms state-of-the-art monocular depth estimation methods on colonoscopy datasets.
• The framework generates depth maps with sharp, anatomically aligned boundaries.
• Achieved temporally consistent predictions and real-time throughput.

Conclusions:

• EndoStreamDepth provides a robust solution for monocular depth estimation in endoscopic video.
• The generated depth maps are essential for advancing robotic surgery and other automated procedures.
• The framework's ability to produce precise and temporally stable depth information marks a significant step forward in surgical visualization.