Recovering dense 3D point clouds from single endoscopic image.

Area of Science:

• Computer Vision
• Medical Imaging
• Deep Learning

Background:

• 3D point cloud reconstruction from monocular endoscopic images is difficult.
• Existing methods often struggle with data quality and completeness.

Purpose of the Study:

• To develop a novel deep learning framework for high-quality 3D point cloud reconstruction from single monocular endoscopic images.
• To address the challenge of incomplete data and defects in reconstructed point clouds.

Main Methods:

• Utilized an unsupervised mono-depth learning network to generate depth maps from monocular endoscopic images.
• Employed a generative Endo-AE network (auto-encoder) to repair defects and complete incomplete 3D point clouds.
• Evaluated performance against state-of-the-art learning-based and non-learning based stereo reconstruction methods.

Main Results:

• The proposed framework significantly outperforms existing methods in 3D point cloud reconstruction.
• The Endo-AE model successfully generates high-quality, dense 3D endoscopic point clouds, even from data with up to 60% missing information.
• Generated and released five in-vivo and two synthetic 3D medical datasets.

Conclusions:

• The developed computational framework effectively produces high-quality, dense 3D point clouds from single endoscopic images.
• This technology has potential applications in augmented reality, virtual reality, and computer-mediated medical procedures.