Path Integral Policy Improvement With Population Adaptation.

Area of Science:

• Reinforcement Learning
• Robotics
• Control Systems

Background:

• Path Integral Policy Improvement (PI²) is an efficient reinforcement learning algorithm for high-dimensional dynamical systems.
• Existing PI² methods require significant user expertise for parameter tuning, impacting performance.
• The efficiency of PI² is highly dependent on the appropriate selection of adjustable parameters.

Purpose of the Study:

• To propose an extension of the PI² algorithm that automatically adjusts critical parameters.
• To reduce the user burden associated with parameter tuning in reinforcement learning.
• To improve the optimization performance of PI² in complex motion acquisition tasks.

Main Methods:

• Developed an extended PI² algorithm with automatic parameter adjustment capabilities.
• Tested the algorithm on motion acquisition tasks for three simulated legged robot types.
• Validated the acquired motion through a real-world robot experiment.

Main Results:

• The proposed method successfully automated critical parameter adjustments in PI².
• Significant improvements in optimization performance were observed compared to existing methods.
• The algorithm effectively acquired motions for simulated legged robots.

Conclusions:

• The automated PI² extension alleviates the need for manual parameter tuning.
• The proposed approach enhances reinforcement learning efficiency and performance in robotic motion acquisition.
• The method's validity is confirmed by successful real-world robot experiments.