Factors of Transferability for a Generic ConvNet Representation

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Convolutional Networks (ConvNets) are leading methods for visual recognition.
• Representation learning via ConvNets is effective for various tasks.
• Transfer learning uses trained ConvNet activations as image representations for new tasks.

Purpose of the Study:

• Investigate factors impacting the transferability of ConvNet representations.
• Optimize these factors to enhance performance on target visual recognition tasks.
• Categorically order tasks by similarity to the source task to understand performance correlations.

Main Methods:

• Analyzed source ConvNet training parameters (architecture, data distribution).
• Examined feature extraction parameters (layer selection, dimensionality reduction).
• Optimized identified factors across 17 visual recognition tasks.

Main Results:

• Significant performance improvements achieved on diverse visual recognition tasks.
• Demonstrated a correlation between task similarity to the source and transfer performance.
• Established a categorical ordering of tasks based on their similarity to the source.

Conclusions:

• Optimizing ConvNet training and feature extraction parameters enhances representation transfer.
• Task similarity to the source task is a key predictor of transfer learning success.
• The findings provide a framework for improving ConvNet-based visual recognition systems.