Real-time surgical tool detection with multi-scale positional encoding and contrastive learning

Area of Science:

• Computer Vision
• Medical Robotics
• Surgical Technology

Background:

• Real-time surgical tool detection is crucial for laparoscopic surgery analysis, trainee evaluation, and robotic system autonomy.
• Current detection methods struggle with processing speed and accuracy, often limited by anchor-based or region proposal techniques.
• Existing non-anchor-based detectors have shown limited success in addressing these limitations.

Purpose of the Study:

• To introduce an advanced anchor-free, transformer-based architecture for real-time surgical tool detection.
• To enhance adaptability to diverse tool appearances and improve detection accuracy and speed in laparoscopic data.

Main Methods:

• Utilized a transformer-based, anchor-free architecture for enhanced detection.
• Incorporated multi-scale features and positional encoding to capture tool context and structure.
• Employed a supervised contrastive loss to optimize object embedding representations for improved classification.

Main Results:

• The proposed method demonstrated superior performance over state-of-the-art (SOTA) techniques.
• Achieved nearly 4% higher mean Average Precision (mAP) compared to the SOTA DSSS method.
• Reduced inference time by 113% and showed a 7% higher mAP than the baseline model.

Conclusions:

• The developed anchor-free, transformer-based approach significantly advances real-time surgical tool detection.
• The method offers improved accuracy, speed, and adaptability, outperforming current SOTA models.
• This work provides a robust foundation for enhancing surgical analytics and autonomous robotic surgery.