Learning explainable task-relevant state representation for model-free deep reinforcement learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• State representations significantly boost deep reinforcement learning (DRL) speed and data efficiency, particularly in visual tasks.
• Task-relevant state representations can further enhance performance by focusing on pertinent features and filtering distractions.
• Current methods for task-relevant representations often rely on model-based DRL, which requires learning a transition function, posing challenges with potential inaccuracies.

Purpose of the Study:

• To propose a novel, direct, and robust method for explainable task-relevant state representation (ETrSR) in model-free DRL.
• To avoid the complexities and potential performance degradation associated with learning transition models.
• To improve the understanding of decision-making processes in DRL through interpretable state representations.

Main Methods:

• Feature disentanglement from states using a beta variational autoencoder (β-VAE).
• Utilizing a reward prediction model to guide feature relevance to the specific task.
• Decoding task-related features to generate explainable states, bypassing the need for a transition model.

Main Results:

• The proposed ETrSR method was validated on the CarRacing environment and DeepMind control suite (DMC) tasks.
• Demonstrated outstanding performance, even in environments with significant distractions.
• Provided explainability, enhancing the understanding of the agent's decision-making process.

Conclusions:

• ETrSR offers an effective approach for generating task-relevant and explainable state representations in model-free DRL.
• The method improves learning efficiency and robustness without requiring a transition model.
• ETrSR facilitates better interpretability and performance in complex DRL environments.