DQNViz: A Visual Analytics Approach to Understand Deep Q-Networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Visualization

Background:

• Deep Q-Network (DQN) models achieve superhuman performance in games like Atari 2600.
• Understanding and interpreting DQN agent behavior during training is challenging due to extensive data and long training times.

Purpose of the Study:

• To introduce DQNViz, a visual analytics system designed to demystify the DQN training process.
• To enable comprehensive analysis of agent experiences and facilitate model interpretation and control.

Main Methods:

• Developed DQNViz, a system offering four levels of detail for visualizing DQN training.
• Focused on Atari games with simple action spaces, specifically Breakout, for initial analysis.
• Extracted action/reward patterns from agent experiences using visual analytics.

Main Results:

• DQNViz provides insights into the 'black box' of DQN training.
• Identified actionable patterns for interpreting and controlling DQN model training.
• Demonstrated effectiveness through case studies with deep learning experts.

Conclusions:

• DQNViz enhances understanding and diagnosis of DQN models.
• The system aids domain experts in potentially improving deep reinforcement learning agent performance.
• Visual analytics is a promising approach for complex AI model interpretability.