Variational policy search using sparse Gaussian process priors for learning multimodal optimal actions.

Area of Science:

• Robotics
• Machine Learning
• Artificial Intelligence

Background:

• Policy search reinforcement learning is crucial for robot control.
• Non-parametric policies, like Gaussian process regression, handle high-dimensional sensor data.
• Existing methods assume unique optimal actions, limiting complex robot manipulations.

Purpose of the Study:

• To develop novel non-parametric policy search algorithms for scenarios with multiple optimal actions.
• To address the limitations of unimodal policy assumptions in complex robotic tasks.
• To enhance the performance of robot control policies in practical applications.

Main Methods:

• Proposed two novel algorithms based on sparse Gaussian process prior and variational Bayesian inference.
• Introduced multimodality to capture multiple optimal actions.
• Developed a mode-seeking approach to select one optimal action from multiple possibilities.
• Utilized multiple overlapped Gaussian processes (GPs) and the student-t distribution.

Main Results:

• Demonstrated the effectiveness of the proposed multimodal and mode-seeking sparse Gaussian process policy search algorithms.
• Successfully applied the algorithms to object manipulation tasks in simulations.
• Showcased the ability to capture multiple optimal actions, overcoming unimodality limitations.

Conclusions:

• The novel approaches effectively handle multiple optimal actions in non-parametric policy searches.
• These methods offer significant improvements for complex robot manipulation tasks.
• The proposed algorithms provide a more robust and flexible framework for reinforcement learning in robotics.