Pact-Net: Parallel CNNs and Transformers for medical image segmentation.

Area of Science:

• Medical Image Analysis
• Computer Vision
• Machine Learning

Background:

• Skin lesion segmentation is crucial for clinical diagnosis but challenging due to low contrast and variable lesion characteristics.
• Traditional deep learning methods struggle with extracting global features, leading to segmentation inaccuracies like over- and under-segmentation.

Purpose of the Study:

• To develop an optimized skin image segmentation model that effectively integrates local and global features.
• To address the limitations of traditional convolutional neural networks (CNNs) in capturing global context for medical image segmentation.

Main Methods:

• Designed Parallel CNNs and Transformers for Medical Image Segmentation (Pact-Net), a dual-branch network capturing local and global features.
• Introduced a novel fusion module (CSMF) utilizing channel, spatial attention, and multi-scale mechanisms to integrate Transformer-extracted global information with CNN-extracted local features.

Main Results:

• Pact-Net achieved superior performance on ISIC 2016, ISIC 2017, and ISIC 2018 datasets, with accuracy rates of 86.95%, 79.31%, and 84.14%, respectively.
• Experiments on cell and polyp datasets demonstrated Pact-Net's robustness, learning, and generalization capabilities.
• Ablation studies confirmed the effectiveness of individual components within Pact-Net.

Conclusions:

• Pact-Net effectively leverages the strengths of CNNs and Transformers for enhanced skin lesion segmentation.
• The proposed model demonstrates state-of-the-art segmentation capabilities, offering significant potential to aid clinicians in disease diagnosis and treatment.