Practical Probabilistic Model-Based Reinforcement Learning by Integrating Dropout Uncertainty and Trajectory...

Area of Science:

• Robotics
• Machine Learning
• Artificial Intelligence

Background:

• Probabilistic model-based reinforcement learning (MBRL) using neural networks faces challenges in prediction stability, accuracy, and control.
• Existing methods struggle to effectively manage system uncertainty in complex control tasks.

Purpose of the Study:

• To propose a novel approach, DPETS (dropout-based probabilistic ensembles with trajectory sampling), to enhance MBRL.
• To improve prediction stability, accuracy, and control capability in neural network-based MBRL.

Main Methods:

• DPETS combines Monte Carlo dropout (MC Dropout) and trajectory sampling for stable system uncertainty prediction.
• A specialized loss function corrects neural network fitting errors for accurate probabilistic model prediction.
• State propagation is extended to filter aleatoric uncertainty, boosting control performance.

Main Results:

• DPETS demonstrated superior performance in Mujoco benchmark tasks and a robot arm manipulation task.
• The method outperformed related MBRL approaches in average return and convergence speed.
• DPETS achieved better results than model-free baselines with significant sample efficiency.

Conclusions:

• DPETS offers a robust solution for enhancing prediction and control in MBRL.
• The proposed method shows significant improvements in stability, accuracy, and sample efficiency.
• DPETS provides a promising advancement for complex robotic control applications.