Stepwise PathNet: a layer-by-layer knowledge-selection-based transfer learning algorithm.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Transfer learning leverages pre-trained neural networks for new tasks, but performance hinges on selecting appropriate network layers.
• Current methods like PathNet require modular networks, limiting the use of widely available non-modular pre-trained models.

Purpose of the Study:

• To introduce Stepwise PathNet, a novel method for automatic layer selection in transfer learning using non-modular pre-trained neural networks.
• To overcome the limitations of existing methods that require modular network architectures.

Main Methods:

• Stepwise PathNet treats individual layers of non-modular networks as modules for selection.
• Automatic layer selection is achieved through a training process within the Stepwise PathNet framework.

Main Results:

• Stepwise PathNet demonstrated superior performance in transfer learning tasks compared to fine-tuning and training from scratch.
• Experiments showed accuracy improvements of up to 8% over fine-tuning and 10% over from-scratch methods.
• The method successfully selected layers not originally supported by PathNet's assumptions.

Conclusions:

• Stepwise PathNet significantly enhances the versatility and effectiveness of transfer learning by enabling automatic selection of layers from non-modular networks.
• This approach broadens the applicability of transfer learning to a wider range of pre-trained models.