Shielded Planning Guided Data-Efficient and Safe Reinforcement Learning.

Area of Science:

• Robotics
• Artificial Intelligence
• Control Theory

Background:

• Safe reinforcement learning (RL) is crucial for general-purpose robotic systems.
• Existing methods primarily focus on post-training safety, neglecting in-training safety and exploration efficiency.

Purpose of the Study:

• To develop a model-based safe RL method ensuring safety during training and improving exploration.
• To introduce Shielded Planning guided Policy Optimization (SPPO) for enhanced robotic learning.

Main Methods:

• SPPO integrates path planning and shielding mechanisms into policy optimization.
• Utilizes Model Predictive Path Integral (MPPI) for shielded exploration and data collection.
• Employs an advantage-based shielding rule and Task-Oriented Parameter Optimization (TOPO) for policy improvement.

Main Results:

• SPPO demonstrates superior policy performance compared to baseline methods.
• Achieves enhanced learning efficiency and safety performance during the training phase.
• Provides explicit theoretical guarantees for training safety, deployment safety, and policy performance.

Conclusions:

• SPPO effectively addresses the challenges of in-training safety and exploration efficiency in RL for robotics.
• The method offers a robust framework for developing safer and more efficient general-purpose robotic systems.
• Theoretical guarantees and experimental results validate the efficacy of SPPO.