M$^{3}$-DEGREES Net: Monocular-Guided Metric Marching Depth Estimation With Graph-Based Relevance Ensemble for...

Area of Science:

• Medical Robotics
• Computer Vision
• Surgical Navigation

Background:

• Robotic endoluminal surgery requires accurate depth perception for gastrointestinal interventions.
• Current monocular camera-based depth estimation methods often lack precision or rely on external sensors.
• Effective visual navigation is crucial for enhancing surgical treatments.

Purpose of the Study:

• To develop a novel monocular vision-guided network for accurate metric depth estimation in robotic endoluminal surgery.
• To improve visual navigation capabilities for surgeons during gastrointestinal interventions.
• To address limitations of existing depth estimation methods in terms of accuracy and reliance on external hardware.

Main Methods:

• Introduced M³-Degrees Net, a graph learning-based network for metric marching depth (MD) estimation.
• Utilized a generative model for initial scale-free depth map generation.
• Employed a relational graph convolutional network with multi-modal visual knowledge fusion for optimized MD prediction.
• Incorporated ego-motion correction and a multi-layer regression network for enhanced accuracy and granularity.

Main Results:

• Achieved an overall metric marching depth error below 27.3%, significantly outperforming existing methods.
• Demonstrated satisfactory performance on public and in-house datasets, including challenging clinical gastrointestinal endoscopy data.
• Validated robustness in environments with varying reflections, such as cavity mucus.

Conclusions:

• M³-Degrees Net offers a promising solution for accurate monocular depth estimation in robotic endoluminal surgery.
• The network enhances visual navigation and has significant clinical potential for gastrointestinal interventions.
• The proposed method overcomes limitations of current approaches, improving surgical precision and safety.