Exploration With Task Information for Meta Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Meta reinforcement learning (meta-RL) enables rapid task adaptation by utilizing prior knowledge.
• Context-based meta-RL improves data efficiency but often suffers from inefficient exploration due to inadequate task information utilization.

Purpose of the Study:

• To propose a novel context-based meta-RL framework with an enhanced exploration mechanism.
• To address the inefficient exploration and execution problem in existing meta-RL methods.

Main Methods:

• Introduced a new objective with two exploration terms to improve action and task embedding space exploration.
• Divided meta-training into task-independent and task-relevant exploration stages based on action information utilization.
• Decoupled task inference and task execution with distinct optimization objectives for each stage.

Main Results:

• The proposed algorithm significantly outperforms popular meta-RL methods on MuJoco benchmarks.
• Demonstrated superior performance in terms of sample efficiency and task achievement in both dense and sparse reward settings.

Conclusions:

• The novel framework effectively enhances exploration in context-based meta-RL.
• The proposed method offers a more efficient way to learn policy and task inference networks, leading to improved overall performance.