Fast Transfer Learning Method Using Random Layer Freezing and Feature Refinement Strategy.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Inductive transfer learning (ITL) utilizes pretrained deep convolutional neural networks (DCNNs).
• Moore-Penrose inverse (MPI)-based parameter fine-tuning of fully connected (FC) layers is a recent ITL approach.
• Current MPI-based ITL methods face challenges due to high computational demands, limiting practical application.

Purpose of the Study:

• To develop a novel fast retraining strategy to enhance the applicability of MPI-based ITL.
• To address the computational bottlenecks of existing MPI-based ITL methods.
• To improve the convergence speed of parameter fine-tuning in DCNNs.

Main Methods:

• Implemented a random layer freezing protocol during retraining epochs to manage feature refinement.
• Incorporated an MPI-based approach for refining FC layer parameters under batch processing.
• Evaluated the strategy on ImageNet pretrained benchmark DCNNs.

Main Results:

• The proposed ITL strategy achieves competitive performance compared to conventional ITL methods.
• Demonstrated significantly improved convergence speed.
• Achieved convergence nearly 1.5 times faster than standard retraining on ResNet-50 using stochastic gradient descent with momentum (SGDM).

Conclusions:

• The novel fast retraining strategy effectively enhances the practicality of MPI-based ITL.
• The method offers a computationally efficient alternative for fine-tuning DCNNs.
• This approach presents a promising direction for accelerating deep learning model adaptation.