MA-SAM: Modality-agnostic SAM adaptation for 3D medical image segmentation.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Computer Vision

Background:

• The Segment Anything Model (SAM) excels in natural image segmentation but struggles with medical images due to domain differences.
• Adapting SAM for medical imaging requires integrating 3D (volumetric/temporal) information while preserving its powerful 2D pre-trained weights.

Purpose of the Study:

• To introduce MA-SAM, a modality-agnostic framework for adapting SAM to diverse 3D medical data (CT, MRI, surgical videos).
• To leverage SAM's 2D backbone by injecting 3D adapters for enhanced feature extraction.

Main Methods:

• MA-SAM employs parameter-efficient fine-tuning, updating minimal weights to preserve SAM's pre-trained knowledge.
• 3D adapters are integrated into the transformer blocks of SAM's image encoder to process volumetric and temporal data.
• The framework is designed to be modality-agnostic, applicable across various medical imaging types.

Main Results:

• MA-SAM consistently outperformed state-of-the-art 3D segmentation methods on five tasks using 11 public datasets (CT, MRI, surgical video).
• Achieved superior Dice scores compared to nnU-Net in CT multi-organ (0.9%), MRI prostate (2.6%), and surgical scene segmentation (9.9%).
• Demonstrated strong generalization capabilities, particularly in challenging tumor segmentation tasks, even with prompts.

Conclusions:

• MA-SAM effectively adapts the Segment Anything Model for 3D medical image segmentation, achieving superior performance.
• The framework's modality-agnostic design and parameter-efficient approach make it a versatile tool for medical AI.
• MA-SAM represents a significant advancement in applying foundation models to complex medical imaging challenges.