Swin Unet3D: a three-dimensional medical image segmentation network combining vision transformer and convolution

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Neuroscience

Background:

• 3D semantic segmentation of brain tumors is crucial for clinical treatment planning using 3D MRI.
• Traditional 3D Convolutional Neural Networks (CNNs) struggle with long-range dependencies due to kernel size limitations.
• Vision Transformers (ViTs) excel at long-range dependencies but have high parameter counts and difficulty with local features in limited data.

Purpose of the Study:

• To develop an advanced deep learning model for accurate and efficient 3D brain tumor segmentation.
• To address the limitations of existing CNNs and ViTs in capturing both local and global image features.
• To improve the understanding of tumor properties like shape and size for better surgical outcomes.

Main Methods:

• Proposed the Swin Unet3D model, a novel architecture for voxel segmentation in medical images.
• Designed a feature extraction sub-module with a parallel structure of Convolution and Vision Transformer (ViT).
• Treated medical image segmentation as a sequence-to-sequence prediction task.

Main Results:

• Achieved Dice coefficients of 0.840 (ET), 0.874 (TC), and 0.911 (WT) on the Brats2021 validation dataset.
• Obtained Dice coefficients of 0.716 (ET), 0.761 (TC), and 0.874 (WT) on the Brats2018 validation dataset.
• Demonstrated a superior balance between model parameters and segmentation accuracy.

Conclusions:

• The Swin Unet3D model effectively combines CNN and ViT advantages for 3D medical image segmentation.
• The proposed model shows significant improvements in brain tumor segmentation accuracy.
• The study offers a promising new tool for enhancing preoperative planning and surgical success rates.