Understanding via Exploration: Discovery of Interpretable Features With Deep Reinforcement Learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Control Systems

Background:

• Understanding complex systems often requires analyzing environmental interactions.
• Deep reinforcement learning (DRL) excels at control but its opaque nature (DNNs) hinders feature relevance identification.
• Mastering unknown systems necessitates understanding which inputs are critical for control.

Purpose of the Study:

• To propose a novel online feature selection framework, dual-world-based attentive feature selection (D-AFS).
• To quantitatively identify the contribution of inputs to the control process in DRL.
• To enhance the interpretability of deep neural networks in control applications.

Main Methods:

• Introduced a dual-world framework with a real and a virtual world possessing distinct features.
• Developed an attention-based evaluation (AR) module for dynamic mapping between the real and virtual worlds.
• Modified existing DRL algorithms to learn within this dual-world environment.

Main Results:

• D-AFS quantitatively identifies feature importance by analyzing DRL responses across both worlds.
• Experiments on four OpenAI Gym classical control systems demonstrated D-AFS's effectiveness.
• D-AFS generated feature combinations comparable or superior to human experts and seven baseline methods.

Conclusions:

• The proposed D-AFS framework effectively addresses the challenge of feature relevance in DRL.
• Selected feature representations by D-AFS correlate strongly with underlying system dynamics.
• D-AFS offers a powerful tool for understanding and improving DRL-based control systems.