ScribFormer: Transformer Makes CNN Work Better for Scribble-Based Medical Image Segmentation.

Area of Science:

• Medical Image Analysis
• Computer Vision
• Artificial Intelligence

Background:

• Convolutional Neural Network (CNN) frameworks with encoder-decoder architectures are standard for scribble-supervised segmentation.
• These methods struggle to capture global shape information due to the local receptive fields of convolutional layers, limiting their effectiveness with sparse scribble annotations.

Purpose of the Study:

• To introduce ScribFormer, a novel CNN-Transformer hybrid model designed to improve scribble-supervised medical image segmentation.
• To address the limitations of existing methods in learning global shape context from limited scribble data.

Main Methods:

• Developed ScribFormer, a triple-branch architecture integrating CNN, Transformer, and an attention-guided class activation map (ACAM) branch.
• Fused local features from CNN with global representations from the Transformer branch.
• Utilized the ACAM branch to unify shallow and deep convolutional features for enhanced performance.

Main Results:

• ScribFormer demonstrated superior performance compared to state-of-the-art scribble-supervised segmentation methods on public and private datasets.
• The proposed method achieved results comparable to or better than fully-supervised segmentation techniques.
• The hybrid approach effectively overcomes the limitations of purely CNN-based methods in capturing global context.

Conclusions:

• ScribFormer offers a significant advancement in scribble-supervised medical image segmentation by effectively combining local and global feature learning.
• The model's ability to leverage limited annotations surpasses current state-of-the-art and even fully-supervised methods, highlighting its potential for practical applications.
• The hybrid CNN-Transformer architecture provides a robust solution for challenging segmentation tasks with sparse supervision.