Semantic-Aware Message Broadcasting for Efficient Unsupervised Domain Adaptation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Vision transformers show promise in computer vision tasks but exhibit poor generalization on out-of-distribution data due to distribution shifts.
• Current unsupervised domain adaptation (UDA) methods using a single global class token for feature alignment lack flexibility and fail to capture regional semantics.

Purpose of the Study:

• To propose Semantic-aware Message Broadcasting (SAMB) for more informative and flexible feature alignment in vision transformers for UDA.
• To enhance the richness of alignment features by enabling semantic-aware adaptive message broadcasting.

Main Methods:

• Introduced learned group tokens to aggregate global information and facilitate adaptive message broadcasting to different semantic regions.
• Studied adversarial-based feature alignment (ADA) and pseudo-label based self-training (PST) for UDA.
• Implemented a two-stage training strategy combining ADA and PST.

Main Results:

• SAMB enables group tokens to learn informative and diverse information for effective domain alignment.
• The proposed two-stage training strategy significantly improves the adaptation capability of vision transformers.
• Experiments on DomainNet, OfficeHome, and VisDA-2017 datasets validate the effectiveness of the proposed methods.

Conclusions:

• Semantic-aware Message Broadcasting (SAMB) effectively addresses the generalization limitations of vision transformers in UDA.
• The combination of SAMB with ADA and PST in a two-stage training approach offers a robust solution for unsupervised domain adaptation.