Related Experiment Video
Updated: May 29, 2025
04:48
Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
Published on: November 30, 2022
2.6K
Sam2Rad: A segmentation model for medical images with learnable prompts
Assefa Seyoum Wahd1, Banafshe Felfeliyan1, Yuyue Zhou1
1Department of Radiology and Diagnostic Imaging, University of Alberta, Canada.
Computers in Biology and Medicine
|February 6, 2025
Summary
Sam2Rad enhances medical image segmentation by enabling Segment Anything Model (SAM) variants to automatically segment bones in ultrasound images without manual prompts, significantly improving accuracy across datasets.
Area of Science:
- Medical Imaging
- Artificial Intelligence
- Computer Vision
Background:
- Segment Anything Model (SAM) and MedSAM show promise for medical image segmentation but require time-consuming manual prompts.
- Existing SAM variants struggle with bone segmentation in ultrasound images due to domain shift, even with manual prompts.
- Manual prompts, whether sparse (boxes, points, text) or dense (masks), are insufficient for accurate bone segmentation in ultrasound.
Purpose of the Study:
- To introduce Sam2 for Radiology (Sam2Rad), a novel framework for automatic bone segmentation in ultrasound images.
- To eliminate the need for manual prompts in medical image segmentation tasks using SAM and its variants.
- To improve the segmentation performance of SAM and SAM2 on bony regions in musculoskeletal ultrasound.
Main Methods:
- Developed a Prompt Predictor Network (PPN) using a lightweight cross-attention mechanism to generate segmentation prompts.
- PPN leverages SAM's image encoder features and learnable embeddings to predict bounding boxes, masks, and high-dimensional embeddings.
- Froze all SAM modules, training only PPN to preserve SAM's knowledge and enable efficient parameter utilization; compatible with any SAM architecture.
Main Results:
- Sam2Rad achieved significant improvements in Dice scores across three musculoskeletal ultrasound datasets (hip, wrist, shoulder).
- Demonstrated up to 51.3% improvement in Dice score for shoulder ultrasound segmentation compared to baseline SAM2.
- Showcased effective performance with minimal labeled data (as few as 10 images) and robustness in zero-shot generalization scenarios.
Conclusions:
- Sam2Rad effectively addresses the limitations of SAM-based models for bone segmentation in ultrasound images.
- The proposed framework enables prompt-free, accurate, and efficient medical image segmentation.
- Sam2Rad offers versatile application modes (fully autonomous, semi-autonomous, manual) and demonstrates broad compatibility with SAM architectures.
