Context-guided entropy minimization for semi-supervised domain adaptation.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Semi-Supervised Domain Adaptation (SSDA) addresses domain shift using labeled source and limited target data.
• Model adaptation, via source pre-training and target fine-tuning, enhances data privacy by avoiding simultaneous data availability.
• Entropy Minimization (EM) is a common SSDA method but can lead to over-confident predictions and performance degradation.

Purpose of the Study:

• To quantitatively analyze the over-confidence issue in Entropy Minimization (EM).
• To propose a novel method, longitudinal self-distillation, to mitigate EM's over-confidence problem.
• To improve the performance and robustness of domain adaptation models.

Main Methods:

• Quantitative analysis of Entropy Minimization (EM) over-confidence.
• Development of longitudinal self-distillation guided by a dynamic teacher label distribution.
• Graph construction on target data for pseudo-label propagation to capture context category dependency.
• Distillation of learned label distributions to refine EM predictions.

Main Results:

• Demonstrated the quantitative impact of over-confidence in EM.
• Validated the effectiveness of longitudinal self-distillation in addressing EM's limitations.
• Achieved improved performance in domain adaptation tasks through the proposed method.

Conclusions:

• Longitudinal self-distillation effectively combats the brute-force over-confidence of EM.
• Capturing label dependencies is crucial for robust domain adaptation.
• The proposed method offers a promising approach for advanced SSDA techniques.