Certifiable Robustness to Adversarial State Uncertainty in Deep Reinforcement Learning

Area of Science:

• Robotics
• Artificial Intelligence
• Control Theory

Background:

• Deep neural networks (DNNs) excel in robotics but lack robustness guarantees for safety-critical applications.
• Adversarial attacks and sensor noise can cause DNNs to make dangerous decisions, as seen in autonomous vehicles.
• Existing defenses offer limited formal guarantees against such perturbations.

Purpose of the Study:

• To develop an online defense mechanism for deep reinforcement learning (DRL) algorithms that provides certified adversarial robustness.
• To ensure reliable decision-making in safety-critical robotic systems despite potential input perturbations.
• To offer a certificate of solution quality for the learned policy, even under uncertainty.

Main Methods:

• Leveraging research on certified adversarial robustness to create an online defense for DRL.
• Computing guaranteed lower bounds on state-action values during execution to select robust actions.
• Implementing and evaluating the defense on a deep Q-network (DQN) policy.

Main Results:

• The proposed defense significantly increases robustness against noise and adversarial examples in pedestrian collision avoidance, classic control tasks, and Atari Pong.
• The approach provides a certificate of solution quality for the DRL policy.
• Extended prior work with new performance guarantees, algorithmic extensions, and adversarial behavior scenarios.

Conclusions:

• The developed online certifiably robust defense enhances the safety and reliability of DRL systems in critical applications.
• This method offers a practical way to achieve formal robustness guarantees for DNNs in robotics.
• The findings pave the way for safer autonomous systems by addressing vulnerabilities to adversarial inputs.