Deep-ELA: Deep Exploratory Landscape Analysis with Self-Supervised Pretrained Transformers for Single- and Multi-Objective Continuous Optimization Problems | JoVE Visualize

Area of Science:

Artificial Intelligence
Optimization
Machine Learning

Background:

Exploratory Landscape Analysis (ELA) features numerically characterize single-objective continuous optimization problems, aiding machine learning tasks like algorithm selection and configuration.
Traditional ELA features exhibit strong correlations and limited applicability to multi-objective optimization problems.
Deep learning approaches, such as point-cloud transformers, have been proposed as alternatives but require substantial labeled training data.

Purpose of the Study:

To propose a hybrid framework, Deep-ELA, that integrates deep learning with ELA features.
To address the drawbacks of existing ELA methods, particularly their limitations in multi-objective optimization and feature correlation.
To develop a method for characterizing both single- and multi-objective continuous optimization problems using deep representations.

Main Methods:

Developed a hybrid approach, Deep-ELA, combining deep learning and ELA features.
Pre-trained four transformers on millions of randomly generated optimization problems.
Learned deep representations of fitness landscapes for continuous single- and multi-objective optimization problems.

Main Results:

Successfully pre-trained transformers to generate deep representations of optimization problem landscapes.
The Deep-ELA framework demonstrates effectiveness for analyzing both single- and multi-objective continuous optimization problems.
The framework can be used out-of-the-box or fine-tuned for specific tasks related to algorithm behavior and problem understanding.

Conclusions:

Deep-ELA offers a powerful hybrid approach, leveraging deep learning and ELA features to overcome limitations of traditional methods.
The pre-trained transformers provide robust deep representations for characterizing complex optimization landscapes.
This framework enhances the analysis and understanding of continuous optimization problems, with potential for broad applications in algorithm selection and configuration.