Bootstrapping of Parameterized Skills Through Hybrid Optimization in Task and Policy Spaces

Area of Science:

• Robotics
• Machine Learning
• Control Theory

Background:

• Modern robotic systems require high adaptability to varying task conditions.
• Traditional reinforcement learning (RL) struggles with rapid adaptation due to extensive data requirements for relearning.
• Incremental learning is crucial for efficient robotic skill acquisition.

Purpose of the Study:

• To develop a method for effective incremental task learning in robots.
• To reduce the number of data rollouts required for robots to adapt to new task parameters.
• To improve the efficiency and feasibility of reinforcement learning in dynamic robotic applications.

Main Methods:

• Proposed a parameterized skill encoded as a meta-learner for generalizing actions.
• Utilized dynamic motion primitives (DMPs) for task-specific parameterization.
• Introduced a hybrid optimization method combining manifold-based coarse optimization and unrestricted parameter search.

Main Results:

• Parameterized skills significantly improve the initialization for incremental task learning.
• The hybrid optimization method reduces the number of required rollouts for adaptation.
• The approach was validated on a 10-DOF planar arm and a humanoid robot point reaching task.

Conclusions:

• Parameterized skills offer a more effective approach to incremental learning in robotics.
• The proposed hybrid optimization enhances the efficiency of robotic adaptation.
• This method addresses key challenges in applying reinforcement learning to real-world robotic tasks.