Multi-task learning with cross-task consistency for improved depth estimation in colonoscopy.

Area of Science:

• Medical Imaging
• Computer Vision
• Gastroenterology

Background:

• Colonoscopy is crucial for detecting colon and rectal abnormalities like ulcers and polyps.
• Accurate 3D reconstruction of the colon requires precise depth estimation, which is challenging due to variable conditions and monocular video.
• Existing computer vision depth estimation methods are not well-quantified for colonoscopy datasets.

Purpose of the Study:

• To develop a novel multi-task learning (MTL) approach for accurate depth estimation in colonoscopy.
• To improve the extraction of salient features for better geometric understanding of the colonic mucosa.
• To benchmark state-of-the-art depth estimation techniques on the C3VD colonoscopy dataset.

Main Methods:

• A multi-task learning (MTL) framework with a shared encoder and two decoders (surface normal and depth estimation).
• Incorporation of attention mechanisms in the depth estimator for enhanced global context awareness.
• Utilizing surface normal prediction as an auxiliary task and applying a cross-task consistency loss.

Main Results:

• The proposed MTL approach demonstrated significant improvements over the state-of-the-art Big-to-Small (BTS) method.
• Achieved a 15.75% improvement in relative error and a 10.7% improvement in δ1.25 accuracy.
• Provided the first benchmark of current depth estimation methods on the C3VD colonoscopy dataset.

Conclusions:

• The novel MTL approach effectively improves depth estimation accuracy in colonoscopy videos.
• Leveraging auxiliary tasks like surface normal prediction enhances geometric feature extraction for depth estimation.
• This work sets a new benchmark for depth estimation in colonoscopy and aids in objective disease burden evaluation.