Risk-Aware Model-Based Control.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Model-Based Reinforcement Learning (MBRL) offers data efficiency but lags in performance on complex, high-dimensional problems compared to model-free methods.
• Existing MBRL methods struggle with real-world uncertainties and performance in high-dimensional state spaces.

Purpose of the Study:

• To propose a novel MBRL algorithm, Risk-Aware Model-Based Control (RAMCO), that addresses performance limitations in complex environments.
• To enhance the robustness and applicability of MBRL for real-world robotic applications.

Main Methods:

• Developed RAMCO, combining uncertainty-aware deep dynamics models with Conditional Value at Risk (CVaR) for risk assessment.
• Utilized a model-free solver for generating warm-up training data to improve performance in both low- and high-dimensional scenarios.

Main Results:

• RAMCO demonstrated superior performance compared to state-of-the-art reinforcement learning algorithms on a walking robot model.
• Evaluated RAMCO using the Eidos environment, highlighting its advantages in handling multi-dimensional, randomly initialized deep neural networks.

Conclusions:

• RAMCO effectively integrates risk awareness and model-based control, outperforming existing methods in complex robotic tasks.
• The proposed method enhances MBRL's practical applicability by addressing data efficiency and performance challenges in high-dimensional environments.