On Automated Source Selection for Transfer Learning in Convolutional Neural Networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Transfer learning enables knowledge transfer between tasks, commonly using pre-trained convolutional neural networks (CNNs).
• Selecting the optimal source CNN significantly impacts target task performance in transfer learning.
• Current methods lack a principled approach for choosing source CNNs, despite numerous pre-trained models.

Purpose of the Study:

• To investigate automatic ranking of source CNNs for efficient transfer learning.
• To develop a principled framework for selecting the best source CNN for a given target task.
• To enable zero-shot ranking of source CNNs without task-specific training.

Main Methods:

• Developed an information-theoretic framework to quantify the source-target task relationship.
• Derived an efficient, zero-shot approach for automatically ranking potential source CNNs.
• Evaluated the ranking method using diverse datasets: PlacesMIT, MNIST, and a real-world MRI database.

Main Results:

• The proposed information-theoretic framework effectively captures the source-target relationship.
• The automatic ranking approach demonstrates significant efficacy in selecting optimal source CNNs.
• Experimental results confirm the practical utility and performance improvements in transfer learning.

Conclusions:

• The developed method provides a principled and efficient way to rank source CNNs for transfer learning.
• This automated ranking significantly enhances the performance of target tasks by selecting appropriate pre-trained models.
• The approach offers a valuable tool for leveraging pre-trained CNNs in various machine learning applications.