SCADA: Sparse cross attention for domain adaptive semantic segmentation

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Unsupervised domain adaptive (UDA) semantic segmentation aims to predict semantic labels for unannotated target images using models trained on annotated source data.
• Existing attention mechanisms in UDA often overlook large-region semantic categories, leading to inefficient computations and wasted resources.
• Current UDA methods frequently neglect the inherent connections within training data, hindering effective pixel discrimination.

Purpose of the Study:

• To propose an efficient sparse cross attention (SCA) block to reduce computational redundancy in UDA semantic segmentation.
• To enhance pixel discrimination by incorporating pixel-wise contrastive learning within the UDA framework.
• To improve the overall performance and efficiency of UDA semantic segmentation models.

Main Methods:

• Developed a sparse cross attention (SCA) block that aggregates contextual information horizontally and vertically to create sparse attention maps.
• Implemented pixel-wise contrastive learning in the latent space of extracted features to promote intra-class compactness and inter-class separability.
• Evaluated the proposed SCA block and contrastive learning strategy on GTA → Cityscapes, Synthia → Cityscapes, and Cityscapes → Dark Zurich benchmarks.

Main Results:

• The SCA block significantly reduces computational resources, leading to lower time and space complexity.
• The proposed method demonstrates substantial performance improvements on three widely-used UDA benchmarks.
• Integrating SCA with existing UDA methods further enhances their performance, showing broad applicability.

Conclusions:

• The proposed SCA block offers an efficient solution for UDA semantic segmentation by addressing computational inefficiencies.
• Pixel-wise contrastive learning effectively improves pixel representation discrimination across domains.
• The SCA block is a versatile component that can be integrated into various UDA frameworks to boost performance.