Improving instrument detection for a robotic scrub nurse using multi-view voting.

Area of Science:

• Robotics
• Computer Vision
• Surgical Technology

Background:

• Surgical instrument detection is crucial for robotic scrub nurses.
• Deep learning models show promise but have performance limitations.
• Real-world application requires high accuracy and reliability.

Purpose of the Study:

• To enhance surgical instrument detection accuracy using a novel approach.
• To demonstrate the effectiveness of combining deep learning with multi-view instance-based voting.
• To improve the reliability of instrument detection in robotic surgery.

Main Methods:

• Collected RGB data and point clouds from multiple viewpoints using a robotic scrub nurse setup.
• Utilized trained Mask R-CNN models to obtain predictions from each view.
• Developed a multi-view voting scheme based on predicted instances to combine data and improve detection.

Main Results:

• Reduced detection errors by over 82% compared to single-view methods.
• Five viewpoints were sufficient on average to accurately infer instrument arrangement.
• The proposed method significantly improved the performance of the instrument detector.

Conclusions:

• The multi-view voting scheme drastically improves instrument detector performance.
• The method is practical for real-time surgical procedures without workflow disruption.
• The implementation and data are publicly available for research.