Related Experiment Video
Updated: Jul 8, 2025
Author Spotlight: Segmentation and VR for Advanced Neurovascular Interventions
Published on: April 5, 2024
Segment anything model for medical images?
Yuhao Huang1, Xin Yang1, Lian Liu1
1National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China.
The Segment Anything Model (SAM) shows promise for medical image segmentation but requires careful prompting and fine-tuning for optimal performance across diverse cases. Its utility in assisting human annotation is also highlighted.
Area of Science:
- Computer Vision
- Medical Imaging Analysis
- Artificial Intelligence
Background:
- The Segment Anything Model (SAM) is a foundational model for general image segmentation.
- Medical image segmentation (MIS) presents unique challenges due to complex modalities, fine structures, and variable object scales.
Purpose of the Study:
- To comprehensively evaluate SAM's performance on medical image segmentation tasks.
- To establish a large-scale, diverse medical image segmentation dataset (COSMOS 1050K) for rigorous validation.
- To analyze various SAM configurations and prompting strategies for MIS.
Main Methods:
- Collected and organized 53 open-source datasets, creating the COSMOS 1050K dataset (18 modalities, 84 objects, 1050K images, 6033K masks).
- Conducted extensive experiments analyzing SAM's performance with different versions (ViT-B, ViT-H) and prompting methods (Everything mode, box prompts, point prompts).
- Evaluated SAM's potential for assisting human annotation and the impact of fine-tuning on specific medical tasks.
Main Results:
- SAM demonstrated variable performance, excelling in some areas but showing instability in others.
- SAM with ViT-H generally outperformed ViT-B, and box prompts were more effective than the Everything mode.
- SAM showed potential for efficient human annotation and performance gains after task-specific fine-tuning (4.39%-6.68% DICE improvement).
Conclusions:
- SAM is a powerful tool for medical image segmentation but requires careful prompt engineering and fine-tuning for robust application.
- The COSMOS 1050K dataset provides a valuable resource for future research in medical image segmentation with foundation models.
- Further research is needed to address SAM's sensitivity to prompt randomness and optimize its use in complex medical imaging scenarios.
