Fast Covariance Matrix Adaptation for Large-Scale Black-Box Optimization.

Area of Science:

• Optimization Algorithms
• Computational Intelligence
• Machine Learning

Background:

• Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is effective for gradient-free optimization.
• Standard CMA-ES struggles with high-dimensional and large-scale problems.
• Efficient scaling of optimization algorithms is crucial for complex applications.

Purpose of the Study:

• To introduce a fast variant of CMA-ES (Fast CMA-ES) designed for large-scale black-box optimization.
• To improve the efficiency and scalability of CMA-ES for high-dimensional search spaces.
• To analyze the theoretical underpinnings and practical performance of the proposed Fast CMA-ES.

Main Methods:

• Approximating the covariance matrix using a low-rank matrix composed of a few vectors.
• Generating new solutions by utilizing two specific vectors from the low-rank approximation.
• Achieving linear internal complexity concerning the search space dimension.

Main Results:

• The Fast CMA-ES demonstrates improved efficiency in terms of running time compared to standard CMA-ES.
• Experimental results show competitive or superior performance against limited-memory CMA-ES variants on large-scale problems.
• The algorithm, when combined with a restart strategy, achieved remarkable performance on CEC'2010 benchmarks, outperforming other large-scale CMA-ES variants.

Conclusions:

• Fast CMA-ES offers a scalable and efficient solution for large-scale black-box optimization problems.
• The low-rank approximation strategy effectively addresses the dimensionality limitations of traditional CMA-ES.
• The proposed method shows significant potential for complex optimization tasks, particularly in high-dimensional settings.