Joint estimation of depth and motion from a monocular endoscopy image sequence using a multi-loss rebalancing network

Area of Science:

• Medical Imaging
• Computer Vision
• Surgical Technology

Background:

• Accurate 3D surface modeling from monocular endoscopy is crucial for enhancing laparoscopic surgery.
• Existing methods face challenges in intuitiveness and precision during surgery.

Purpose of the Study:

• To propose a novel multi-loss rebalancing method for joint depth and motion estimation from monocular endoscopy image sequences.
• To improve the accuracy and robustness of 3D reconstruction for clinical applications.

Main Methods:

• Utilized feature descriptors for monitoring depth and motion estimation networks.
• Incorporated epipolar constraints and neighborhood spatial information for enhanced depth estimation.
• Employed reprojection information and multi-view relative pose fusion for camera motion reconstruction.
• Defined relative response loss, feature consistency loss, and epipolar consistency loss for unsupervised learning.

Main Results:

• Achieved significant reductions in motion estimation error (42.1%, 53.6%, 50.2% across three scenes).
• Obtained an average 3D reconstruction error of 6.456 ± 1.798 mm.
• Demonstrated reliable depth estimation and trajectory reconstruction for endoscopy images.

Conclusions:

• The proposed method effectively generates reliable 3D surface models from monocular endoscopy.
• This technology offers meaningful applications for improving clinical laparoscopic surgery.
• The unsupervised learning-based approach enhances accuracy and robustness in surgical imaging.