AutoTune: Automatically Tuning Convolutional Neural Networks for Improved Transfer Learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Transfer learning leverages pre-trained deep networks for tasks with limited data.
• Fine-tuning the last layers of pre-trained models is standard but layers may be suboptimal for new tasks.

Purpose of the Study:

• To introduce a novel mechanism for automatically tuning Convolutional Neural Networks (CNNs) to improve transfer learning.
• To address the limitations of standard fine-tuning by optimizing pre-trained layers for target tasks.

Main Methods:

• Utilized Bayesian Optimization to tune pre-trained CNN layers with target data knowledge.
• Implemented a greedy criteria approach, automatically tuning the CNN based on validation data performance.
• Replaced the final softmax layer neurons with the number of target task classes before tuning.

Main Results:

• The proposed AutoTune method achieved superior classification accuracy compared to standard transfer learning baselines.
• Achieved 95.92% accuracy on CalTech-101, 86.54% on CalTech-256, and 84.67% on Stanford Dogs.
• Demonstrated that tuning pre-trained CNN layers with target dataset knowledge enhances transfer learning capabilities.

Conclusions:

• The AutoTune method effectively improves transfer learning by optimizing pre-trained CNN layers.
• Automatic tuning using Bayesian Optimization offers a significant advantage over standard fine-tuning techniques.
• The findings highlight the importance of adapting pre-trained model components to specific target datasets for optimal performance.