Related Experiment Video
Updated: Jun 26, 2025
06:18
Author Spotlight: Segmentation and VR for Advanced Neurovascular Interventions
Published on: April 5, 2024
1.0K
Surgical-DeSAM: decoupling SAM for instrument segmentation in robotic surgery
Yuyang Sheng1,2, Sophia Bano1,2, Matthew J Clarkson1,3
1Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK.
Summary
Surgical-DeSAM enables real-time surgical instrument segmentation without manual prompts by using automatic bounding box generation. This method significantly improves upon state-of-the-art techniques in robotic surgery.
Area of Science:
- Robotics
- Computer Vision
- Medical Imaging
Background:
- The Segment Anything Model (SAM) shows promise for image segmentation but requires manual prompts.
- Prompting is infeasible in real-time surgical applications due to lack of per-frame annotations and high annotation costs.
Purpose of the Study:
- To develop Surgical-DeSAM, a system for automatic instrument segmentation in robotic surgery.
- To enable real-time segmentation without manual prompting, overcoming SAM's limitations in surgical contexts.
Main Methods:
- Utilized DETR (Detection Transformer) for automatic bounding box prompt generation of surgical instruments.
- Employed a decoupled SAM (DeSAM) architecture, replacing the image encoder with DETR's.
- Fine-tuned prompt encoder and mask decoder, incorporating Swin-transformer for enhanced feature representation.
Main Results:
- Validated Surgical-DeSAM on EndoVis 2017 and 2018 datasets.
- Achieved Dice metrics of 89.62% (EndoVis 2017) and 90.70% (EndoVis 2018).
- Demonstrated superior performance compared to state-of-the-art (SOTA) instrument segmentation methods.
Conclusions:
- Surgical-DeSAM provides real-time instrument segmentation in robotic surgery without manual prompts.
- The proposed method significantly outperforms existing SOTA segmentation techniques.
