Deep Neural Network Parameter Selection via Dataset Similarity Under Meta-Learning Framework.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep neural networks (DNNs) performance is highly sensitive to hyperparameter selection and weight initialization.
• Current methods often optimize these factors independently, limiting model adaptability and effectiveness.

Purpose of the Study:

• To develop a novel meta-learning framework for joint recommendation of hyperparameters and initial weights.
• To leverage dataset similarity for improved deep learning model optimization.

Main Methods:

• Extracting meta-features (shallow and deep) from historical datasets.
• Computing dataset similarity in a meta-feature space for query datasets.
• Recommending parameter configurations based on similar historical datasets.

Main Results:

• The framework was evaluated on 105 real-world image classification tasks.
• Demonstrated consistent outperformance over state-of-the-art baselines for both vision transformers and convolutional neural networks.
• Validated the effectiveness of dataset-driven parameter recommendation.

Conclusions:

• Jointly recommending hyperparameters and initial weights via dataset similarity is effective.
• The proposed meta-learning framework enhances deep learning model performance.
• This dataset-driven approach offers a more adaptable and effective optimization strategy.