Policy search in continuous action domains: An overview

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Continuous action policy search is a key area in AI research.
• Recent advances in deep reinforcement learning and evolutionary algorithms have spurred progress.

Purpose of the Study:

• To provide a comprehensive survey of continuous action policy search methods.
• To offer a unified perspective on diverse approaches.
• To explore the relationships between different policy search families.

Main Methods:

• Literature review and synthesis of policy search methodologies.
• Categorization of methods including deep reinforcement learning, evolutionary algorithms, Bayesian optimization, and directed exploration.
• Analysis of factors contributing to sample efficiency.

Main Results:

• A unified framework is presented, connecting various policy search techniques.
• Key relationships between different families of methods are elucidated.
• Factors influencing the sample efficiency of policy search algorithms are identified.

Conclusions:

• Understanding the connections between policy search methods is crucial.
• Sample efficiency is a critical consideration for practical applications.
• This survey provides a foundation for future research in continuous control.