Spatio-temporal layers based intra-operative stereo depth estimation network via hierarchical prediction and...

Area of Science:

• Computer Vision
• Medical Robotics
• Artificial Intelligence

Background:

• Intra-operative depth estimation is crucial for enhancing robotic surgery safety via augmented vision and motion constraints.
• Accurate 3D scene positioning is essential for advanced surgical applications.
• Deep learning offers significant potential for improving depth estimation in endoscopic views.

Purpose of the Study:

• To develop a deep learning-based approach for accurate 3D depth estimation in intra-operative endoscopic scenes.
• To enhance the precision of 3D spatial information recovery for surgical environments.
• To improve the safety and capabilities of robot-assisted surgery through advanced vision.

Main Methods:

• A fully 3D encoder-decoder network integrating spatio-temporal layers was designed.
• The network utilizes hierarchical prediction and progressive learning strategies.
• This approach aims to boost prediction accuracy and reduce training duration.

Main Results:

• The proposed network achieved a Mean Absolute Error (MAE) of 2.55±1.51 mm and Root Mean Square Error (RMSE) of 5.23±1.40 mm.
• Performance was evaluated on 8 surgical videos with 1280×1024 resolution.
• The method outperformed six other state-of-the-art techniques on the same dataset.

Conclusions:

• The developed network demonstrates promising depth estimation performance for intra-operative stereo images.
• This technology can be integrated into robot-assisted surgery systems.
• The enhanced depth perception contributes to improved surgical safety.