CoTrFuse: a novel framework by fusing CNN and transformer for medical image segmentation

Area of Science:

• Medical Image Analysis
• Artificial Intelligence
• Deep Learning

Background:

• Medical image segmentation is vital but challenging for deep learning models like U-Net.
• Convolutional Neural Networks (CNNs) struggle with global context, while Transformers lack local detail.
• Existing methods face limitations in capturing both global and remote semantic information.

Purpose of the Study:

• To propose a novel network, CoTrFuse, that combines CNN and Transformer strengths for enhanced medical image segmentation.
• To address the limitations of restricted receptive fields in CNNs and limited local information acquisition in Transformers.

Main Methods:

• Developed CoTrFuse, a network utilizing EfficientNet and Swin Transformer as dual encoders.
• Implemented a Swin Transformer and CNN Fusion module to integrate features before skip connections.
• Evaluated CoTrFuse on the ISIC-2017 and COVID-QU-Ex datasets.

Main Results:

• CoTrFuse demonstrated superior performance compared to several state-of-the-art medical image segmentation methods.
• Experimental results validated the network's effectiveness on diverse medical imaging datasets.

Conclusions:

• The proposed CoTrFuse network effectively combines CNN and Transformer architectures for superior medical image segmentation.
• CoTrFuse offers a promising advancement in medical image analysis, outperforming existing techniques.