Learning Transferable Parameters for Unsupervised Domain Adaptation.

Area of Science:

• Machine Learning
• Computer Vision

Background:

• Unsupervised domain adaptation (UDA) aims to adapt models from labeled source to unlabeled target domains despite distribution shifts.
• Deep neural networks in UDA focus on learning domain-invariant features for better generalization.
• Domain-specific information in current UDA methods can hinder model performance on target domains.

Purpose of the Study:

• To propose a novel method, Transferable Parameter Learning (TransPar), to enhance UDA by focusing on essential parameters.
• To mitigate the negative impact of domain-specific information on model generalizability.
• To improve the memorization of domain-invariant information during the adaptation process.

Main Methods:

• Inspired by the lottery ticket hypothesis, we identify 'transferable' parameters crucial for domain invariance and 'untransferable' parameters fitting domain-specific details.
• TransPar dynamically categorizes parameters into transferable and untransferable based on distribution discrepancy during training.
• Separate update rules are applied to transferable and untransferable parameters to optimize their respective roles.

Main Results:

• TransPar significantly outperforms existing methods on image classification and keypoint detection tasks.
• The proposed method demonstrates improved generalization capabilities in unsupervised domain adaptation.
• Experiments confirm the effectiveness of distinguishing and treating parameters differently based on their transferability.

Conclusions:

• Transferable Parameter Learning (TransPar) offers an effective approach to improve unsupervised domain adaptation by selectively updating parameters.
• The method is versatile and can be integrated into various deep UDA architectures.
• TransPar shows promise for handling diverse data distribution shifts in real-world applications.