Deep Transfer Learning Method Using Self-Pixel and Global Channel Attentive Regularization.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Transfer learning is widely applied but susceptible to knowledge loss when adapting to new datasets.
• Existing regularization methods based on knowledge distillation aim to mitigate this issue.
• Feature map alignment is a key technique within knowledge distillation for transfer learning.

Purpose of the Study:

• To propose a novel transfer learning regularization method.
• To address knowledge loss issues in transfer learning using knowledge distillation.
• To enhance classification accuracy on target datasets.

Main Methods:

• A new transfer learning regularization method based on feature map alignment.
• Integration of two attention-based submodules: self-pixel attention (SPA) and global channel attention (GCA).
• SPA jointly considers source and target model features; GCA assesses channel importance across all layers.

Main Results:

• The proposed method demonstrated improved overall classification accuracy compared to existing techniques.
• Experiments were conducted on commonly used datasets to validate the method's effectiveness.
• The combined SPA and GCA submodules were crucial for the observed performance gains.

Conclusions:

• The novel transfer learning regularization method effectively reduces knowledge loss.
• Feature map alignment combined with attention mechanisms offers a promising direction for transfer learning.
• The proposed method enhances model performance in classification tasks across diverse datasets.