Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Deep reinforcement learning (DRL) and deep multiagent reinforcement learning (MARL) show promise in AI, autonomous vehicles, and robotics.
• A key limitation is sample inefficiency, requiring millions of interactions, hindering real-world deployment.
• The exploration problem, or efficiently gathering informative experiences, is a major bottleneck, especially in complex environments.

Purpose of the Study:

• To provide a comprehensive survey of exploration methods in single-agent and multiagent reinforcement learning.
• To systematically classify existing exploration approaches.
• To empirically compare different exploration methods for DRL and identify future research directions.

Main Methods:

• Categorization of exploration methods into uncertainty-oriented and intrinsic motivation-oriented approaches.
• Inclusion of other notable exploration techniques.
• Algorithmic analysis and a unified empirical comparison of DRL exploration methods on standard benchmarks.

Main Results:

• Identification of key challenges in efficient exploration for DRL and MARL.
• Systematic review and classification of existing exploration strategies.
• Empirical evaluation highlighting the performance of various exploration techniques.

Conclusions:

• Exploration remains a critical challenge in DRL and MARL.
• The survey provides a foundation for understanding and advancing exploration strategies.
• Future research should focus on addressing open problems and developing more efficient exploration techniques.