Evolving Multimodal Robot Behavior via Many Stepping Stones with the Combinatorial Multiobjective Evolutionary...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Multimodal problems in reinforcement learning require agents to adapt behavior based on context.
• Curricula, or ordered subtasks, aid in solving complex problems but effective ordering is challenging.
• Existing methods for optimizing multiple subtasks simultaneously include NSGA-II, NSGA-III, and ε-Lexicase Selection.

Purpose of the Study:

• To introduce and investigate the Combinatorial Multiobjective Evolutionary Algorithm (CMOEA) for solving multimodal reinforcement learning problems.
• To compare CMOEA's performance against established multi-objective optimization algorithms.
• To explore methods for improving multi-objective optimization, including linear combinations of objectives and leveraging auxiliary objectives.

Main Methods:

• The study introduces and implements the Combinatorial Multiobjective Evolutionary Algorithm (CMOEA).
• CMOEA was compared against NSGA-II, NSGA-III, and ε-Lexicase Selection on diverse problems.
• Experiments included function optimization, simulated robot locomotion, and simulated robot maze navigation.

Main Results:

• CMOEA demonstrated superior or competitive performance across tested multimodal problems.
• Adding a linear combination of objectives enhanced the performance of control algorithms.
• CMOEA showed improved ability to leverage auxiliary objectives in multimodal locomotion tasks.

Conclusions:

• CMOEA is a promising and effective algorithm for addressing challenges in multimodal reinforcement learning.
• The findings suggest that exploring all subtask combinations simultaneously is beneficial.
• Further research into objective combination strategies can improve reinforcement learning agent capabilities.