Learned graphical models for probabilistic planning provide a new class of movement primitives

Area of Science:

• Robotics
• Computational Neuroscience
• Machine Learning

Background:

• Biological movement generation exhibits modularity, stochastic optimality, and learning efficiency.
• Current motor skill learning often uses dynamical systems for movement primitives (MPs), indirectly defining trajectories.
• An alternative MP representation is needed to better align with biological movement control principles.

Purpose of the Study:

• To propose a new movement primitive representation based on probabilistic inference in learned graphical models.
• To integrate stochastic optimal control (SOC) methods within MPs for intrinsic probabilistic planning.
• To learn system dynamics and cost function parameters using reinforcement learning (RL).

Main Methods:

• Developed an MP representation parameterized by a graphical model encoding dynamics and intrinsic cost functions.
• Integrated stochastic optimal control (SOC) principles within the movement primitives.
• Utilized reinforcement learning (RL) to learn model parameters and control policies.
• Evaluated the approach on a challenging 4-link balancing task.

Main Results:

• The proposed probabilistic MP representation significantly facilitates learning.
• The approach demonstrates improved generalization to new task settings without requiring re-learning.
• Inference in the graphical model successfully yields the desired control policy.

Conclusions:

• The novel MP representation, leveraging probabilistic graphical models and SOC, offers a more biologically plausible and efficient approach to motor skill learning.
• This method enhances robotic control by improving learning speed and adaptability.
• The findings suggest a promising direction for developing more sophisticated and versatile robotic movement systems.