Frame-Correlation Transfers Trigger Economical Attacks on Deep Reinforcement Learning Policies

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Adversarial attacks are crucial for evaluating reinforcement learning (RL) policies before deployment.
• Existing gradient-based attacks are computationally intensive, perturbing all pixels.
• Gradient-free methods offer selective perturbations but ignore temporal correlations, increasing complexity.

Purpose of the Study:

• To investigate the exploitation of frame-transferability for generating minimal yet potent adversarial attacks in image-based RL.
• To introduce novel frame-correlation transfer (FCT) mechanisms to enhance adversarial attack generation efficiency.

Main Methods:

• Developed three types of frame-correlation transfers (FCTs): anterior case transfer, random projection-based transfer, and principal components-based transfer.
• Utilized a genetic algorithm for adversary generation incorporating FCTs.
• Evaluated FCTs on four state-of-the-art RL policies across six Atari games.

Main Results:

• FCTs dramatically accelerate adversarial attack generation compared to existing methods, often reducing computation time to near zero.
• Demonstrated a clear trade-off between the computational complexity and the potency of the FCT mechanisms.
• Showcased the feasibility of real-time adversarial attacks in image-based RL environments.

Conclusions:

• Frame-correlation transfers are effective in creating computationally efficient and powerful adversarial attacks for RL.
• This research highlights the practical threat of real-time adversarial attacks in reinforcement learning.
• The proposed FCTs offer a more realistic and time-efficient approach to RL policy evaluation.