Collaborative Hyperparameter Recommendation by Coupled Matrix Factorization With Kernel.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Hyperparameter selection is crucial for machine learning model performance.
• Existing meta-learning methods for hyperparameter recommendation have limitations in flexibility and applicability.
• Current approaches often rely on either meta-features or partial hyperparameter evaluations exclusively.

Purpose of the Study:

• To propose a novel hyperparameter recommendation framework overcoming limitations of existing methods.
• To flexibly integrate meta-features and partial hyperparameter evaluations.
• To enhance the accuracy and applicability of hyperparameter recommendation.

Main Methods:

• Developed a coupled matrix factorization (CMF) framework.
• Employed a dual-component model: matrix completion for partial evaluations and an auxiliary module for meta-features.
• Incorporated kernel-based techniques to handle metadata nonlinearity.

Main Results:

• The CMF framework successfully integrates meta-features and partial evaluations.
• The dual-component model captures complex patterns missed by single-method approaches.
• Kernel-based techniques improved recommendation accuracy by addressing nonlinearity.

Conclusions:

• The proposed CMF framework offers a flexible and effective solution for hyperparameter recommendation.
• The method demonstrates superior performance compared to existing meta-learning recommendation models and search algorithms.
• The approach is validated on diverse datasets using Support Vector Machines (SVMs) and Vision Transformers (ViTs).