VSS-SAM++: Visual State Space-Aware SAM for 3D Medical Image Segmentation

Area of Science:

• Medical Imaging
• Computer Vision
• Artificial Intelligence

Background:

• Segment Anything Model (SAM) shows promise in natural image segmentation but struggles with medical imaging due to data distribution shifts and 3D complexities.
• Existing SAM-based methods using parameter-efficient transfer learning (PETL) often fail to capture essential 3D contextual information for volumetric segmentation.

Purpose of the Study:

• To introduce VSS-SAM++, a novel dual-branch architecture designed to improve 3D medical image segmentation.
• To leverage SAM's visual priors and Vision Mamba's long-range dependency modeling for enhanced segmentation accuracy.

Main Methods:

• Developed a dual-branch architecture combining SAM as the primary encoder with a parallel Mamba branch for cross-slice dependency modeling.
• Implemented a gated hybrid attention module to dynamically fuse features from both branches, adaptively weighting multi-view representations.
• Evaluated the framework on nine public CT and MRI datasets for multi-organ and lesion segmentation tasks.

Main Results:

• VSS-SAM++ achieved superior performance, outperforming existing methods by 0.2-11.3% in Dice score across diverse medical imaging datasets.
• Demonstrated significant improvements in segmentation precision by minimizing feature ambiguity through adaptive feature fusion.
• Showcased robustness to domain shifts and scalability across different imaging modalities (CT and MRI).

Conclusions:

• VSS-SAM++ offers a significant advancement in 3D medical image segmentation by effectively integrating 2D and 3D contextual information.
• The framework's adaptability and performance suggest strong potential for clinical deployment in medical image analysis.
• This approach addresses key limitations of applying general segmentation models to specialized medical imaging domains.