Domain Adaptive and Generalizable Network Architectures and Training Strategies for Semantic Image Segmentation

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Unsupervised domain adaptation (UDA) and domain generalization (DG) are crucial for applying machine learning models to new, unlabeled datasets.
• Existing UDA and DG semantic segmentation methods often rely on outdated network architectures.
• Challenges include overfitting to source domains and preserving fine details or long-range context due to image manipulation for memory efficiency.

Purpose of the Study:

• To benchmark recent network architectures, including Transformers, for UDA and DG semantic segmentation.
• To design and introduce the DAFormer network tailored for UDA and DG tasks.
• To propose HRDA, a multi-resolution framework to overcome limitations of low-resolution or cropped images in UDA/DG.

Main Methods:

• Benchmarking recent architectures like Transformers for UDA/DG.
• Developing DAFormer with three strategies: Rare Class Sampling, Thing-Class ImageNet Feature Distance, and learning rate warmup.
• Introducing HRDA, a multi-resolution framework using high-resolution crops for details and low-resolution crops for context with learned scale attention.

Main Results:

• DAFormer and HRDA significantly improve state-of-the-art performance in UDA and DG semantic segmentation.
• Performance gains exceed 10 mIoU across 5 diverse benchmarks.
• The proposed methods effectively mitigate source domain bias and enhance context understanding.

Conclusions:

• DAFormer and HRDA represent significant advancements in UDA and DG for semantic segmentation.
• The developed strategies effectively address overfitting and memory constraints.
• The multi-resolution approach in HRDA successfully balances detail preservation and context capture for robust domain generalization.