Efficacy of Modern Neuro-Evolutionary Strategies for Continuous Control Optimization

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Continuous control optimization is crucial for robotics and AI.
• Neuro-evolutionary strategies offer a promising alternative to traditional reinforcement learning.
• Evaluating these strategies requires diverse and complex benchmark problems.

Purpose of the Study:

• To analyze the efficacy and scalability of modern neuro-evolutionary strategies for continuous control.
• To compare the performance of different neuro-evolutionary algorithms, particularly OpenAI-ES.
• To investigate the compatibility of reward functions across evolutionary and reinforcement learning paradigms.

Main Methods:

• Utilized a wide range of qualitatively different benchmark problems for evaluation.
• Assessed scalability with respect to problem complexity and parameter count.
• Compared OpenAI-ES against other leading neuro-evolutionary algorithms.
• Analyzed the transferability of reward functions between evolutionary and reinforcement learning.

Main Results:

• Neuro-evolutionary methods demonstrate general effectiveness and good scalability.
• OpenAI-ES consistently outperformed or matched other algorithms across benchmarks.
• Hyper-parameter settings showed relative robustness for these strategies.
• Reward functions optimized for one method class were often ineffective for the other.

Conclusions:

• Neuro-evolutionary strategies, especially OpenAI-ES, are highly effective for continuous control.
• The efficacy of comparative studies between evolutionary and reinforcement learning is questioned due to reward function incompatibility.
• Future research should consider method-specific reward function design for unbiased evaluation.