Small is Beautiful: Compressing Deep Neural Networks for Partial Domain Adaptation

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep learning (DL) models require extensive labeled data, making the data labeling process costly.
• Partial domain adaptation (PDA) addresses scenarios where target domain labels are a subset of source domain labels.
• Existing PDA methods often incur significant computational overhead due to model complexity.

Purpose of the Study:

• To develop a unified framework integrating partial domain adaptation (PDA) and model compression.
• To reduce the computational overhead associated with deep PDA models.
• To maintain or improve classification performance while enhancing model efficiency.

Main Methods:

• A novel training process seamlessly combines PDA and model compression.
• Minimization of soft-weighted maximum mean discrepancy (SWMMD) to reduce cross-domain distribution divergence.
• Gradient statistics and batch normalization (BN) layer scaling factors are utilized for channel pruning and model compression.

Main Results:

• The proposed method achieves classification performance comparable to state-of-the-art PDA techniques.
• Significant reductions in model size and computational overhead were observed.
• The approach effectively compresses overparameterized models by pruning redundant channels.

Conclusions:

• Integrating PDA with model compression offers an efficient solution for deep learning.
• The method successfully addresses the computational challenges in partial domain adaptation.
• This work presents a practical approach for deploying efficient deep learning models in resource-constrained environments.