Value-Regularized Reinforcement Learning for Model Predictive Control of Autonomous Mobile Robots Under Stochastic...

Area of Science:

• Robotics
• Control Systems
• Artificial Intelligence

Background:

• Autonomous mobile robots require robust control for reliable operation.
• Model Predictive Control (MPC) is challenged by process disturbances and model mismatch.
• Reinforcement Learning-based Model Predictive Control (RL-MPC) offers adaptive control but can be sensitive to nominal parameters.

Purpose of the Study:

• To introduce a novel control strategy, value-regularized RL-MPC (VR-RLMPC), to improve the robustness of autonomous robot control.
• To enhance RL-MPC by incorporating isotropic state regularization into the stage cost without increasing computational complexity.
• To evaluate the performance of VR-RLMPC against RL-MPC and other baselines under various model mismatch conditions.

Main Methods:

• Introduced isotropic state regularization into the RL-MPC stage cost, creating VR-RLMPC.
• Utilized Lyapunov-drift analysis to establish theoretical performance bounds under specific assumptions.
• Conducted simulations on linear and nonlinear nonholonomic vehicle systems to compare VR-RLMPC with RL-MPC and temporal-difference baselines.

Main Results:

• VR-RLMPC demonstrated approximately 25% lower settling-step indices compared to RL-MPC.
• Worst-case performance degradation for VR-RLMPC was 2.8%, significantly lower than conventional MPC baselines (25-49%).
• VR-RLMPC outperformed a stochastic temporal-difference baseline in nominal settings.

Conclusions:

• VR-RLMPC offers improved robustness and performance for autonomous robot control, especially under model mismatch.
• The method effectively reduces settling times and performance degradation without added online optimization complexity.
• VR-RLMPC presents a promising advancement for reliable autonomous systems operating in uncertain environments.