Occlusion-robust markerless surgical instrument pose estimation.

Area of Science:

• Robotics
• Computer Vision
• Surgical Technology

Background:

• Accurate surgical instrument pose estimation is crucial for Robot-assisted Minimally Invasive Surgery (RMIS) navigation and autonomous task execution.
• Current methods struggle with partial visibility, occlusions, and dynamic surgical environments, limiting their reliability.

Purpose of the Study:

• To propose a robust, markerless, vision-based framework for estimating the 6DoF pose of surgical instruments.
• To address challenges posed by partial visibility and occlusions in surgical scenes.

Main Methods:

• A keypoint object representation combined with a PnP solver for stable and accurate pose computation.
• A novel mask-based data augmentation technique to improve model learning under occlusion.
• Generation of a new dataset with high-accuracy ground truth for instrument pose estimation.

Main Results:

• The proposed network achieves submillimeter accuracy in instrument pose estimation.
• Experimental results demonstrate the framework's generalizability across various occlusion types and surgical instruments.
• The system effectively handles partial tool visibility and occlusions.

Conclusions:

• The developed vision-based framework offers a significant improvement for markerless surgical instrument pose estimation in RMIS.
• The approach enhances robustness against common challenges like occlusions, paving the way for more reliable robotic surgery.
• The method shows potential for integration into real-world surgical navigation and autonomous robotic systems.