Continual Reinforcement Learning for Quadruped Robot Locomotion.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Continuous learning is essential for robot intelligence and autonomy.
• Continual reinforcement learning (RL) aims to enable robots to learn sequentially (plasticity) while retaining past knowledge (stability).
• Existing RL methods struggle with catastrophic forgetting and loss of plasticity in sequential tasks.

Purpose of the Study:

• To develop a continual reinforcement learning method for quadruped robots that balances plasticity and stability.
• To minimize modifications to the original RL learning process.
• To overcome catastrophic forgetting and maintain performance on previously learned tasks.

Main Methods:

• Utilizes the Piggyback algorithm to identify and protect task-specific parameters.
• Reinitializes unused parameters to enhance plasticity.
• Encourages policy network exploration through entropy maximization in the soft network.

Main Results:

• The proposed method enables sequential learning in robots without significant performance degradation on prior tasks.
• Demonstrates superior stability and less disruption to RL training compared to traditional continual learning algorithms.
• Experimental validation on robot locomotion tasks confirms the method's effectiveness.

Conclusions:

• The developed continual RL approach successfully addresses the plasticity-stability dilemma in robot learning.
• The method offers a stable and efficient way to improve robot intelligence and autonomy through sequential task acquisition.
• This work provides a robust solution for continual learning in real-world robotic applications.