Strategic Evolutionary Reinforcement Learning With Operator Selection and Experience Filter.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Reinforcement Learning
• Evolutionary Algorithms

Background:

• Shared replay buffers are crucial for synergy in Evolutionary Reinforcement Learning (ERL).
• Existing ERL methods suffer from objective conflicts between evolutionary algorithms and reinforcement learning, degrading replay buffer quality.
• This conflict hinders the effective learning and performance of ERL agents.

Purpose of the Study:

• To propose a novel strategic ERL algorithm (SERL-OS-EF) that resolves objective conflicts.
• To enhance the synergy between evolutionary population dynamics and reinforcement learning agents.
• To improve the overall quality and utility of the shared replay buffer.

Main Methods:

• Implemented an operator selection strategy to boost individual performance and experience quality.
• Introduced an experience filter to maintain long-term buffer quality by removing suboptimal data.
• Developed a dynamic mixed sampling strategy to optimize RL agent learning efficiency from the buffer.

Main Results:

• Demonstrated superior performance of SERL-OS-EF in challenging MuJoCo locomotion and Ant-Maze environments with deceptive rewards.
• Validated the method's effectiveness in improving replay buffer quality and agent learning.
• Confirmed the practical significance on a low-carbon multi-energy microgrid energy management task.

Conclusions:

• The proposed SERL-OS-EF effectively addresses objective conflicts in ERL, enhancing replay buffer quality and agent performance.
• The strategic combination of operator selection, experience filtering, and dynamic sampling leads to improved synergy and learning efficiency.
• SERL-OS-EF shows promise for real-world applications, including energy management in microgrids.