Data-Driven Safe Policy Optimization for Black-Box Dynamical Systems With Temporal Logic Specifications.

Area of Science:

• Robotics
• Control Theory
• Artificial Intelligence

Background:

• Learning-based policy optimization shows promise for general-purpose control systems.
• Existing methods face challenges in achieving complex objectives and ensuring safety in black-box systems during learning and execution.

Purpose of the Study:

• To develop a novel reinforcement learning (RL)-based method, Data-driven Safe Policy Optimization (D2SPO), for safe policy improvement in black-box systems.
• To jointly learn a control barrier function (CBF) for system safety and a linear temporal logic (LTL) guided RL algorithm for complex task objectives.

Main Methods:

• D2SPO learns a provably safe CBF for black-box dynamical systems by redesigning datasets and loss functions.
• An LTL-guided RL policy is developed to efficiently complete tasks with LTL objectives, leveraging LTL's capability in representing task progress.

Main Results:

• D2SPO demonstrates superior performance compared to state-of-the-art baselines in numerical and experimental studies.
• The method achieves over 95% safety rate and nearly 100% task completion rates.

Conclusions:

• D2SPO effectively addresses the challenges of achieving complex task objectives while ensuring policy safety in black-box systems.
• The developed method offers a robust solution for safe and efficient reinforcement learning in complex control applications.