Distillation-SAM: Knowledge Distillation-Based Auto-Prompt Embedding Learning for Surgical Image Segmentation.

Area of Science:

• Computer Vision
• Medical Imaging
• Surgical Technology

Background:

• Surgical image segmentation is crucial for surgical procedures but current deep learning methods have limitations in generalizability and primarily focus on instrument segmentation.
• The Segment Anything Model (SAM) shows strong generalization in natural images but requires high-quality prompts and is not designed for multi-class medical image segmentation.

Purpose of the Study:

• To develop an effective method, Distillation-SAM, for accurate surgical image segmentation without user-provided prompts.
• To adapt SAM for multi-class semantic segmentation in surgical and medical images.

Main Methods:

• Distillation-SAM freezes SAM's encoder and decoder, introducing a trainable adapter branch for auto-prompt embedding and feature enrichment.
• A direct knowledge distillation constraint guides the learning of auto-prompt embeddings using ground-truth masks.
• A trainable Multilayer Perceptron is incorporated into SAM's decoder to enable multi-class segmentation.

Main Results:

• Distillation-SAM achieves accurate segmentation of surgical objects like vessels, instruments, and tissues.
• Experiments on IVIS, EndoVis2017, and Cholecseg8k datasets show Distillation-SAM outperforms existing methods.
• The method demonstrates effective adaptation of SAM for surgical image segmentation without user prompts.

Conclusions:

• Distillation-SAM offers a robust solution for prompt-less, multi-class surgical image segmentation.
• The proposed method significantly improves segmentation performance across various surgical datasets.
• This work advances the application of foundation models like SAM in the medical domain.