Related Experiment Video
Updated: Sep 10, 2025
Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery
Published on: August 12, 2021
Collaborative surgical instrument segmentation for monocular depth estimation in minimally invasive surgery
Xue Li1, Wenxin Chen2, Xingguang Duan1
1the School of Mechatronical Engineering, Beijing Institute of Technology, 30 Xueyuan Road, Haidian District, Beijing, 100081, China; the Key Laboratory of Biomimetic Robots and Systems, Beijing Institute of Technology, 30 Xueyuan Road, Haidian District, Beijing, 100081, China.
This study introduces a novel two-stage self-supervised framework for monocular depth estimation in surgery. By integrating instrument segmentation, it significantly improves 3D perception for image-guided procedures.
Area of Science:
- Medical Imaging
- Computer Vision
- Surgical Technology
Background:
- Accurate 3D perception is crucial for image-guided surgery, especially in minimally invasive settings.
- Monocular depth estimation from 2D images presents challenges in complex surgical environments.
Purpose of the Study:
- To develop a self-supervised monocular depth estimation framework enhanced by instrument segmentation.
- To improve spatial understanding and geometric reasoning in surgical scenes.
Main Methods:
- A two-stage self-supervised learning approach was proposed.
- Stage 1: Separate training of segmentation and depth estimation models.
- Stage 2: Fusing segmentation masks with RGB input for depth refinement using a shared encoder and multiple decoders.
Main Results:
- The framework was validated on RIS, SCARED, dVPN, and SERV-CT datasets.
- Segmentation-aware depth estimation demonstrated improved geometric reasoning.
- Enhanced performance in challenging scenarios like occlusions and specularities was observed.
Conclusions:
- The proposed framework effectively enhances monocular depth estimation through task-level priors.
- Integrating instrument segmentation improves 3D perception for surgical navigation.
- The approach shows generalizability across diverse surgical datasets.
