SETA: Semantic-Aware Edge-Guided Token Augmentation for Domain Generalization.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Domain generalization (DG) aims to improve model robustness against unseen data domains.
• Current data augmentation methods for DG are often suboptimal for token-based architectures like Vision Transformers (ViT) and Multi-Layer Perceptrons (MLP).
• Existing augmentation techniques primarily focus on Convolutional Neural Networks (CNNs), neglecting the unique characteristics of token-based models.

Purpose of the Study:

• To address the limitations of existing augmentation methods in domain generalization for token-based models.
• To propose a novel augmentation technique that enhances the learning of holistic shape information.
• To improve the robustness and generalization capabilities of ViT and MLP models in the context of domain shifts.

Main Methods:

• Proposing Semantic-aware Edge-guided Token Augmentation (SETA), a novel method for domain generalization.
• SETA perturbs local edge cues while preserving global shape features in token representations.
• Developing stylized variants of SETA combined with state-of-the-art style augmentation techniques for enhanced generalization.

Main Results:

• SETA demonstrates superior performance compared to existing methods on token-based architectures.
• The proposed method effectively enhances the model's ability to learn holistic shape information.
• Experiments across five benchmarks show SETA achieving state-of-the-art results on various ViT and MLP models.
• Theoretical analysis confirms SETA's effectiveness in reducing the generalization risk bound.

Conclusions:

• SETA significantly improves domain generalization for token-based models by focusing on shape information.
• The method offers a robust solution for enhancing model performance under domain shifts.
• SETA represents a promising advancement in data augmentation strategies for modern deep learning architectures.