Safe Reinforcement Learning With Dual Robustness.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Reinforcement learning (RL) agents are susceptible to adversarial attacks, compromising performance and safety.
• Existing safe RL and robust RL methods address these issues in isolation, leaving a gap for unified solutions.
• The challenge lies in balancing feasibility and optimality under adversarial conditions.

Purpose of the Study:

• To develop a systematic framework unifying safe and robust reinforcement learning.
• To address the intertwined challenges of feasibility and optimality in adversarial environments.
• To create a reinforcement learning agent that is both safe and performs optimally under adversarial disturbances.

Main Methods:

• Formulation of the problem as constrained two-player zero-sum Markov games.
• Proposal of a dual policy iteration scheme to simultaneously optimize task and safety policies.
• Development of a practical deep reinforcement learning algorithm, dually robust actor-critic (DRAC), using adversarial networks.

Main Results:

• The dual policy iteration scheme converges to optimal policies for both task performance and safety.
• DRAC demonstrates high performance and persistent safety across various adversarial scenarios (no adversary, safety adversary, performance adversary).
• DRAC significantly outperforms existing baseline methods on safety-critical benchmarks.

Conclusions:

• The proposed framework effectively unifies safe and robust reinforcement learning.
• DRAC offers a practical and effective solution for developing agents that are both safe and robust in adversarial settings.
• This work advances the development of reliable AI systems in safety-critical applications.