Morphological symmetry-aware generalized policy network for deep reinforcement learning.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Morphological symmetry in robots (e.g., humanoids, quadrupeds) offers potential for enhanced robot learning.
• Current deep reinforcement learning (DRL) methods utilize symmetry via data augmentation, equivariant MLPs, and MARL, but DRL training remains unstable due to exploration-driven data distribution shifts.

Purpose of the Study:

• To develop a general-purpose DRL framework for morphologically symmetric robots that ensures stable and robust learning.
• To address the inherent instability in DRL training by incorporating symmetry principles.

Main Methods:

• Proposed a framework modeling the environment as a symmetric Markov decision process (MDP).
• Constructed a full-body policy from a single-sided base policy using symmetry operators.
• Introduced a symmetric Proximal Policy Optimization (PPO) objective with a coupled importance-sampling ratio.

Main Results:

• The proposed method demonstrated superior performance on most symmetric robotic tasks compared to existing approaches.
• Maintained comparable or improved performance over standard PPO on asymmetric tasks.
• The symmetric PPO objective aligns policy optimization with imposed symmetry, offering an alternative to MAPPO-style formulations.

Conclusions:

• The symmetry-assisted DRL framework provides a stable and robust approach for learning control policies in morphologically symmetric robots.
• This method effectively leverages robot morphology to enhance learning efficiency and performance, particularly in symmetric scenarios.