Transformer-Based Visual Segmentation: A Survey.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Visual segmentation partitions images/videos into meaningful groups for applications like autonomous driving and medical analysis.
• Deep learning methods have advanced visual segmentation significantly over the last decade.
• Transformers, originally for NLP, now excel in vision tasks, surpassing convolutional and recurrent networks.

Purpose of the Study:

• To provide a comprehensive overview of transformer-based visual segmentation methods.
• To summarize recent advancements and unify recent transformer-based approaches under a meta-architecture.
• To explore specific subfields and identify future research directions.

Main Methods:

• Review of background, including problem definitions, datasets, and prior convolutional methods.
• Summary of a unifying meta-architecture for transformer-based visual segmentation.
• Examination of various method designs, modifications, and applications based on the meta-architecture.

Main Results:

• Transformers offer robust, unified, and simpler solutions for diverse segmentation tasks.
• Detailed examination of specific subfields: 3D point cloud segmentation, foundation model tuning, domain-aware, efficient, and medical segmentation.
• Re-evaluation of reviewed methods on established datasets.

Conclusions:

• Transformer-based methods represent a significant advancement in visual segmentation.
• The survey highlights key challenges and proposes future research avenues in this rapidly evolving field.