Gradient Harmonization in Unsupervised Domain Adaptation.

Area of Science:

• Machine Learning
• Computer Vision
• Artificial Intelligence

Background:

• Unsupervised domain adaptation (UDA) aims to transfer knowledge from labeled source domains to unlabeled target domains.
• Current UDA methods often optimize domain alignment and classification simultaneously, overlooking inherent conflicts in gradient-based optimization.

Purpose of the Study:

• To address the conflict between domain alignment and classification tasks in UDA.
• To introduce Gradient Harmonization (GH and GH++) as novel solutions to mitigate these optimization conflicts.

Main Methods:

• GH modifies gradient angles to resolve task conflicts, promoting coordinated optimization.
• GH++ further refines this by adjusting gradient angles to minimize deviation from original optimization directions.
• Gradient harmonization strategies are integrated into a dynamically weighted loss function for efficiency.

Main Results:

• GH and GH++ effectively mitigate the conflict between domain alignment and classification tasks.
• The proposed methods are orthogonal to UDA and seamlessly integrate with existing models.
• Experimental results show enhancements in popular UDA baselines and state-of-the-art models.

Conclusions:

• Gradient Harmonization offers a robust approach to improving UDA performance.
• The methods provide theoretical insights and practical improvements for knowledge transfer across domains.