Hyperparameter Recommendation Integrated With Convolutional Neural Network.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Meta-learning shows promise for hyperparameter recommendation.
• Existing meta-learners struggle with complex data features and deep relationships.
• Traditional models lack the capacity to capture intricate data properties.

Purpose of the Study:

• To propose novel hyperparameter recommendation approaches using convolutional neural networks (CNNs).
• To develop a meta-learning framework capable of learning complex features from dataset characteristics and hyperparameter performance.
• To enhance the accuracy and effectiveness of automated hyperparameter tuning.

Main Methods:

• Formulated hyperparameter recommendation as a regression problem, using dataset characteristics as predictors and historical hyperparameter performance as responses.
• Developed a CNN-based learning model with feature selection capabilities.
• Introduced a convolutional denoising autoencoder (ConvDAE) to leverage the spatial structure of the hyperparameter performance space.
• Established a comprehensive two-branch CNN model integrating dataset characteristics and partial evaluations for flexible application.

Main Results:

• Extensive experiments conducted on 400 real classification problems using the Support Vector Machine (SVM).
• Proposed CNN-based approaches demonstrated superior performance compared to existing meta-learning baselines.
• Outperformed various traditional search algorithms in hyperparameter recommendation tasks.
• Validated the high effectiveness of deep learning, specifically CNNs, in this domain.

Conclusions:

• Deep learning, particularly CNNs, offers a powerful solution for hyperparameter recommendation.
• The proposed methods effectively capture complex relationships, leading to improved recommendation accuracy.
• This work advances the field of automated machine learning by providing more sophisticated meta-learning strategies.