Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Deep Learning

Background:

• Loss function design is critical for model performance in machine learning.
• Current methods often rely on handcrafted loss functions, limiting optimization potential.
• Loss function learning is an emerging field aiming to automate this process.

Purpose of the Study:

• To propose a novel meta-learning framework for learning model-agnostic loss functions.
• To enhance supervised learning task performance through automated loss function discovery.
• To validate the framework's versatility across diverse tasks and architectures.

Main Methods:

• A hybrid neuro-symbolic search approach is employed for loss function discovery.
• Evolution-based methods are used to search for symbolic mathematical operations.
• End-to-end gradient-based optimization refines the learned loss functions.

Main Results:

• Meta-learned loss functions outperform standard cross-entropy loss.
• The proposed method surpasses existing state-of-the-art loss function learning techniques.
• Superior performance is demonstrated across various neural network architectures and datasets.

Conclusions:

• The developed meta-learning framework effectively discovers high-performing, model-agnostic loss functions.
• This approach offers a promising direction for improving machine learning model optimization.
• The method shows broad applicability in supervised learning scenarios.