Context-Based Meta-Reinforcement Learning With Bayesian Nonparametric Models.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Deep reinforcement learning (RL) requires extensive data for task completion.
• Meta-reinforcement learning (meta-RL) accelerates adaptation to new tasks using prior experience.
• Current meta-RL methods struggle with non-parametric tasks and non-stationary environments.

Purpose of the Study:

• To develop a meta-RL framework capable of discovering diverse, non-parametric tasks.
• To enable zero-shot adaptation to new tasks, improving data efficiency.
• To enhance robustness in non-stationary environments.

Main Methods:

• Proposed MEta-reinforcement Learning with Task Self-discovery (MELTS).
• Introduced a Dirichlet Process Mixture Model-Variational Autoencoder (DPMM-VAE) for adaptive task clustering and representation learning.
• Implemented a zero-shot adaptation mechanism and recurrence-based context encoding.

Main Results:

• MELTS adaptively discovers multi-modal, non-parametric task distributions.
• Achieved superior sample efficiency and asymptotic performance on continuous control tasks.
• Demonstrated improved performance in non-stationary environments compared to state-of-the-art algorithms.

Conclusions:

• MELTS effectively addresses limitations of existing meta-RL methods in handling non-parametric tasks and non-stationary environments.
• The DPMM-VAE framework enables self-adaptive task discovery and representation.
• MELTS offers a promising direction for more generalizable and efficient reinforcement learning agents.