Robot Policy Improvement With Natural Evolution Strategies for Stable Nonlinear Dynamical System

Area of Science:

• Robotics
• Machine Learning
• Control Theory

Background:

• Robot learning via kinesthetic teaching offers a way to replicate human actions.
• However, complex tasks with limited data present challenges due to error accumulation in imitation learning.

Purpose of the Study:

• To enhance the robustness and adaptability of imitation learning.
• To address limitations in complex task performance with small datasets.

Main Methods:

• A hierarchical learning strategy was proposed, separating low-level behavioral cloning (supervised learning) and high-level policy improvement.
• A Gaussian mixture model (GMM)-based dynamical system encoded demonstrated motion.
• Lyapunov stability theorem ensured global stability conditions for GMM parameters.
• Exponential natural evolution strategies optimized dynamical system parameters for variable impedance control, ensuring stability during exploration.

Main Results:

• The proposed method demonstrated improved robustness and adaptability in robot learning.
• Empirical evaluations on manipulators showed success in motion planning with obstacle avoidance and stiffness learning.

Conclusions:

• The hierarchical learning strategy effectively overcomes compounding errors in imitation learning.
• The approach guarantees global stability, enhancing the reliability of robot learning for complex tasks.