Class-Aware Adversarial Transformers for Medical Image Segmentation

Area of Science:

• Computer Vision
• Medical Image Analysis
• Artificial Intelligence

Background:

• Transformers show promise in medical image analysis for long-range dependency modeling.
• Current transformer models face limitations in feature capture, multi-scale representation, and segmentation accuracy.

Purpose of the Study:

• To introduce CASTformer, an adversarial transformer model designed to overcome limitations in 2D medical image segmentation.
• To improve the accuracy and feature representation capabilities of transformer-based medical image segmentation.

Main Methods:

• Utilized a pyramid structure for multi-scale feature representation and variation handling.
• Developed a class-aware transformer module to learn discriminative object regions with semantic structures.
• Employed an adversarial training strategy with a transformer-based discriminator for enhanced feature capture.

Main Results:

• CASTformer achieved significant improvements over state-of-the-art transformer-based methods on three benchmarks.
• Demonstrated absolute Dice score improvements ranging from 2.54% to 5.88% compared to previous models.
• Qualitative experiments highlighted model transparency and the benefits of transfer learning for reduced dataset size.

Conclusions:

• CASTformer offers a superior approach to 2D medical image segmentation compared to existing transformer models.
• The model's multi-scale and class-aware design effectively addresses limitations in feature representation and accuracy.
• CASTformer shows potential as a foundational model for various downstream medical image analysis tasks.