Real-time surgical instrument detection in robot-assisted surgery using a convolutional neural network cascade

Area of Science:

• Computer Vision
• Robotics
• Medical Imaging

Background:

• Surgical instrument detection is crucial for robot-assisted surgery systems.
• Current deep learning methods often struggle with low detection speed and single-tool focus.

Purpose of the Study:

• To develop a real-time, accurate multi-tool detection method for robot-assisted surgery videos.
• To overcome the speed limitations of existing single-tool detection approaches.

Main Methods:

• A novel frame-by-frame detection approach using a cascading convolutional neural network (CNN).
• Integration of an hourglass network for heatmap generation and a modified VGG network for bounding-box regression.
• Joint prediction of tool tip localization using heatmaps and RGB image frames.

Main Results:

• The proposed method demonstrates superior performance compared to mainstream detection techniques.
• Achieved higher accuracy and speed in surgical instrument detection.
• Validated on the EndoVis Challenge and ATLAS Dione datasets.

Conclusions:

• The cascading CNN approach offers an effective solution for real-time multi-tool detection in robot-assisted surgery.
• The method significantly improves both the accuracy and speed of surgical instrument identification.
• This advancement has the potential to enhance the capabilities of robotic surgical systems.