Diagonal Acceleration for Covariance Matrix Adaptation Evolution Strategies.

Area of Science:

• Optimization Algorithms
• Evolutionary Computation
• Machine Learning

Background:

• Covariance Matrix Adaptation Evolution Strategies (CMA-ES) are powerful but can be slow on ill-conditioned or separable problems.
• Existing CMA-ES variants offer improvements but often introduce new limitations or complexities.

Purpose of the Study:

• To introduce an acceleration technique, adaptive diagonal decoding (dd-CMA), for CMA-ES.
• To enhance CMA-ES performance on nonseparable and coordinate-wise ill-conditioned problems.
• To combine the benefits of default and separable CMA-ES without their respective drawbacks.

Main Methods:

• Introduced a diagonal matrix for adaptive diagonal decoding (dd-CMA) to learn coordinate-wise variances.
• Developed a mechanism to modulate the learning rate based on the correlation matrix's condition number to maintain performance on nonseparable problems.
• Proposed methods for ensuring positive definiteness of the covariance matrix and revised default parameter settings for large dimensions.

Main Results:

• dd-CMA-ES demonstrated significant performance and scaling improvements over standard CMA-ES on functions with coordinate-wise ill-conditioning, up to dimension 5120.
• The method achieved overadditive speedups on certain nonseparable test functions.
• Improvements were observed even for large population sizes.

Conclusions:

• Adaptive diagonal decoding (dd-CMA) effectively accelerates CMA-ES, combining advantages of different variants.
• The proposed method offers a robust improvement for handling complex, real-world optimization landscapes.
• The contributions are complementary and can be integrated into existing CMA-ES frameworks for enhanced performance.