3D bi-directional transformer U-Net for medical image segmentation

Area of Science:

• Medical Imaging
• Deep Learning
• Computer Vision

Background:

• Deep convolutional neural networks (DCNNs) are widely used for image segmentation but struggle with global feature relations.
• Existing methods to improve DCNNs' global reasoning often compromise performance or local feature extraction capabilities.

Purpose of the Study:

• To propose a novel attention mechanism for 3D computation.
• To introduce a new end-to-end segmentation framework (3DTU) for 3D medical image segmentation.
• To enhance the ability of DCNNs to process global imaging features.

Main Methods:

• Designed a novel attention mechanism tailored for 3D computation.
• Developed the 3DTU framework, incorporating a 3D transformer in the encoder and a 3D DCNN in the decoder.
• Processed images in an end-to-end manner, performing 3D computation on both encoder and decoder components.

Main Results:

• The 3DTU framework was evaluated on two independent datasets comprising 3D MRI and CT images.
• Experimental results demonstrated superior performance compared to several state-of-the-art segmentation methods.
• The proposed method achieved significant improvements across various segmentation metrics.

Conclusions:

• The novel attention mechanism and 3DTU framework effectively address the limitations of DCNNs in handling global imaging features.
• The proposed method offers a robust and accurate solution for 3D medical image segmentation.
• 3DTU demonstrates the potential of integrating transformers and DCNNs for advanced medical image analysis.