AISCT-SAM: Customized SAM-Med2D with 3D Context Awareness and Self-Prompt Generation for Fully Automatic Acute...

Lesion segmentation of acute ischemic stroke (AIS) patients on Non-Contrast CT (NCCT) scans plays a crucial role in rapid diagnosis and treatment planning. The direct application of promising semi-automatic SAM-Med2D to AIS lesion segmentation on NCCT scans: 1) does not achieve good performance; 2) cannot incorporate 3D volumetric context; and 3) requires time-consuming prompts and lacks effective domain-specific semantic prompts. Therefore, we propose a novel SAM-Med2D-based model (AISCT-SAM) with 3D context awareness and self-prompt generation for automatic AIS lesion segmentation. First, to better handle the high heterogeneity of AIS lesions between different subjects, we propose the adapter and low-rank adaptation with gate layer which updates parameters on both aspects of the token-mixer and channel-mixer for more comprehensive and flexible fine-tuning. Second, we introduce a plug-and-play depth adapter to extract richer 3D contextual information, while enhancing the local priors that the Transformer architecture lacks. Then, to enable fully automatic segmentation, we propose a self-prompt generator that generates multi-level semantic prompts by leveraging bilateral hemisphere differences. Finally, we devise a multi-level semantic prompt-guided mask decoder that fully takes into account the characteristics of NCCT images. AISCT-SAM was evaluated on the public AISD dataset, a private AIS dataset, and an external AIS dataset. Experimental results show that AISCT-SAM achieves Dice scores of 63.32%, 48.75%, and 43.81% on the three datasets, respectively, surpassing 20 state-of-the-art methods. Additionally, volumetric analysis and external validation suggest that AISCT-SAM could provide valuable reference information for AIS diagnosis and treatment decisions. Furthermore, AISCT-SAM also performs well on a public MRI dataset, indicating its potential generalization capability for other modalities.