Enhancing Stability of Probabilistic Model-Based Reinforcement Learning by Adaptive Noise Filtering

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Current probabilistic model-based reinforcement learning (MBRL) methods face challenges with stability and efficiency due to imperfect models.
• Model bias and prediction noise can negatively impact policy learning and overall performance.

Purpose of the Study:

• To introduce Stabilized Model-Based Policy Optimization (SMBPO) for improved stability and efficiency in MBRL.
• To address noise and bias issues inherent in probabilistic model-based approaches.

Main Methods:

• SMBPO adaptively refines dimensions with abnormal prediction distributions to stabilize probabilistic model training.
• It clips predicted states and estimated value functions to mitigate model bias effects on policy learning.
• Batch Normalization (BN) is integrated to enhance learning efficiency.

Main Results:

• Evaluations on MuJoCo control benchmarks and a dexterous hand task demonstrated SMBPO's effectiveness.
• SMBPO achieved a 90% reduction in training time compared to baselines.
• The method resulted in 50% more cumulative rewards than state-of-the-art model-free and MBRL approaches.

Conclusions:

• SMBPO offers a stable and efficient solution for model-based reinforcement learning.
• The technique significantly enhances learning speed and cumulative rewards.
• SMBPO extends the practical applicability of MBRL in complex robotic control scenarios.