Reward-Adaptive Reinforcement Learning: Dynamic Policy Gradient Optimization for Bipedal Locomotion.

Area of Science:

• Robotics
• Artificial Intelligence
• Control Systems

Background:

• Controlling non-statically bipedal robots presents significant challenges due to complex dynamics and multi-criterion optimization.
• Deep reinforcement learning (DRL) has shown promise but typically uses summed rewards, which can be insufficient for complex hybrid reward scenarios.
• Existing DRL methods struggle to effectively extract information from multiple reward channels simultaneously.

Purpose of the Study:

• To propose a novel reward-adaptive reinforcement learning method for bipedal locomotion.
• To enable simultaneous optimization of control policies by multiple criteria using a dynamic mechanism.
• To address the limitations of summed-reward approaches in DRL for robotic control.

Main Methods:

• Introduced a hybrid and dynamic policy gradient (HDPG) approach for bipedal robot control.
• Employed a multi-head critic to learn separate value functions for each reward component, generating hybrid policy gradients.
• Incorporated dynamic weighting to allow different priorities for optimizing each reward component.

Main Results:

• The proposed HDPG method demonstrated superior performance compared to summed-up-reward approaches in bipedal locomotion tasks.
• The method showed successful transferability from simulation to physical robots.
• Experiments in MuJoCo validated the effectiveness and generalization capabilities of HDPG.

Conclusions:

• The HDPG method offers a more efficient and effective approach to optimizing bipedal robot control policies with multiple criteria.
• This reward-adaptive strategy enhances learning efficiency and policy performance in complex robotic locomotion.
• HDPG represents a significant advancement for DRL applications in robotics, particularly for legged robots.