Meta-Reinforcement Learning in Non-Stationary and Dynamic Environments

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Deep reinforcement learning (DRL) agents require extensive data, unlike humans.
• Meta-reinforcement learning (meta-RL) aims to improve learning efficiency from limited experience.
• Existing meta-RL methods are often restricted to stationary environments and narrow task distributions.

Purpose of the Study:

• To develop a meta-RL training strategy for non-stationary environments.
• To enhance task representation learning using Gaussian mixture models for clustered distributions.
• To enable efficient zero-shot adaptation and continual learning in robotic control tasks.

Main Methods:

• Introduced a new training strategy for meta-RL agents.
• Utilized Gaussian mixture models for task representation learning.
• Evaluated the method on continuous robotic control benchmarks.

Main Results:

• Achieved competitive performance and superior sample efficiency in stationary environments with zero-shot adaptation.
• Demonstrated successful adaptation in non-stationary and continual learning settings.
• Learned distinct behaviors and well-structured task representations from diverse task distributions.

Conclusions:

• The proposed meta-RL strategy effectively addresses limitations of previous methods.
• The approach enables robust learning in dynamic environments and complex task distributions.
• This work advances the capabilities of artificial agents in real-world applications.