Deep-ELA: Deep Exploratory Landscape Analysis with Self-Supervised Pretrained Transformers for Single-Objective and...

Area of Science:

• Artificial Intelligence
• Optimization
• Machine Learning

Background:

• Exploratory Landscape Analysis (ELA) features numerically characterize single-objective optimization problems.
• ELA features are crucial for machine learning tasks like algorithm selection but have drawbacks such as feature correlation and limited applicability to multiobjective problems.
• Deep learning approaches have been proposed as alternatives but require substantial labeled data.

Purpose of the Study:

• To propose a hybrid approach, Deep-ELA, that combines the strengths of deep learning and ELA features.
• To develop a method for characterizing both single- and multiobjective continuous optimization problems.
• To create a framework that can be used out-of-the-box or fine-tuned for specific tasks.

Main Methods:

• Pre-training four transformers on millions of randomly generated optimization problems.
• Learning deep representations of continuous optimization problem landscapes.
• Developing a hybrid deep learning and ELA framework.

Main Results:

• Deep-ELA effectively learns deep representations of optimization landscapes.
• The framework demonstrates applicability to both single- and multiobjective continuous optimization problems.
• The approach addresses limitations of traditional ELA features and data-hungry deep learning methods.

Conclusions:

• Deep-ELA offers a powerful, flexible framework for analyzing and understanding continuous optimization problems.
• The hybrid approach enhances the characterization of optimization landscapes, benefiting algorithm selection and configuration.
• This work paves the way for more advanced analysis of complex optimization tasks.