Self-Supervised monocular depth and ego-Motion estimation in endoscopy: Appearance flow to the rescue

Area of Science:

• Computer Vision
• Medical Imaging
• Robotics

Background:

• Self-supervised learning for depth and ego-motion estimation from monocular videos is effective in autonomous driving.
• A key assumption is constant image brightness, which is violated in endoscopic scenes due to illumination variations and reflections.
• These brightness inconsistencies degrade the accuracy of depth and ego-motion estimation.

Purpose of the Study:

• To address the challenge of brightness inconsistency in endoscopic videos for self-supervised depth and ego-motion estimation.
• To introduce a novel concept, appearance flow, to handle variations in brightness patterns.
• To develop a unified self-supervised framework for simultaneous monocular depth and ego-motion estimation in endoscopic scenes.

Main Methods:

• Developed a unified self-supervised framework incorporating structure, motion, appearance, and correspondence modules.
• Introduced appearance flow to account for brightness variations, enabling a generalized dynamic image constraint.
• The framework aims to accurately reconstruct appearance and calibrate image brightness.

Main Results:

• The proposed unified framework significantly outperforms existing self-supervised approaches on the SCARED and EndoSLAM datasets.
• The framework demonstrates strong generalization ability, performing well on unseen datasets (SERV-CT, Hamlyn) without fine-tuning.
• Superior results highlight the effectiveness of the appearance flow concept and the unified framework.

Conclusions:

• The novel appearance flow concept effectively addresses brightness inconsistency in endoscopic videos.
• The unified self-supervised framework provides accurate simultaneous depth and ego-motion estimation in challenging endoscopic environments.
• The framework exhibits robust generalization capabilities, making it suitable for various clinical applications and camera systems.