Sparse-to-dense coarse-to-fine depth estimation for colonoscopy.

Area of Science:

• Medical Imaging
• Computer Vision
• Robotics

Background:

• Colonoscopy is crucial for colon cancer screening but has limitations in visualization.
• Current methods face challenges due to narrow perspectives and limited depth perception.
• Enhanced 3D visualization can improve diagnostic accuracy and surgical planning.

Purpose of the Study:

• To develop a novel sparse-to-dense, coarse-to-fine depth estimation solution for colonoscopic scenes.
• To integrate direct SLAM (Simultaneous Localization and Mapping) with deep learning for dense depth map generation.
• To reconstruct accurate and detailed 3D colon models for improved clinical insights.

Main Methods:

• Utilized direct SLAM to obtain scattered 3D points from colonoscopic data.
• Employed a deep learning-based depth completion network to generate dense depth maps from sparse data and RGB images.
• Implemented a reconstruction system with photometric error optimization and mesh modeling for enhanced 3D model accuracy.

Main Results:

• Successfully generated accurate, full-resolution dense depth maps from sparse SLAM outputs.
• The deep learning network effectively extracted texture, geometry, and structure features for depth recovery.
• The reconstruction system produced highly accurate 3D colon models with detailed surface textures.

Conclusions:

• The proposed sparse-to-dense, coarse-to-fine strategy significantly improves depth estimation performance.
• Direct SLAM and deep learning-based depth estimation were smoothly fused into a complete dense reconstruction system.
• This approach offers a promising solution for overcoming colonoscopy's visualization limitations in clinical practice.