Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep reinforcement learning (RL) agents excel at complex control tasks but often act as black boxes, hindering user trust.
• Existing interpretation methods for vision-based RL agents frequently fail to capture crucial temporal causal information, limiting their reliability.

Purpose of the Study:

• To develop a novel temporal-spatial causal interpretation (TSCI) model for understanding the long-term behavior of RL agents.
• To address the limitations of current methods by uncovering temporal causal relationships in sequential decision-making.

Main Methods:

• The proposed TSCI model formulates temporal causality to represent causal relations between observations and decisions in RL agents.
• A dedicated causal discovery network identifies temporal-spatial causal features, adhering to temporal causality constraints.
• The model is designed for recurrent agents and efficient causal feature discovery post-training.

Main Results:

• Empirical results demonstrate the TSCI model's ability to generate high-resolution attention masks.
• These masks effectively highlight task-relevant temporal-spatial information crucial for RL agent decision-making.
• The method provides valuable temporal perspective causal interpretations for vision-based RL agents.

Conclusions:

• The TSCI model offers a reliable method for interpreting the behavior of vision-based RL agents.
• It enhances user trust by providing clear insights into the agent's sequential decision-making process.
• The temporal causal interpretations are essential for advancing the practical application of RL.