Curiosity model policy optimization for robotic manipulator tracking control with input saturation in uncertain...

Area of Science:

• Robotics
• Artificial Intelligence
• Control Theory

Background:

• Robot control in uncertain environments with input saturation is challenging for existing methods.
• Model-based reinforcement learning (MBRL) and traditional controllers exhibit limitations in optimal performance.
• There is a need for advanced algorithms to improve tracking accuracy and generalization.

Purpose of the Study:

• To propose a novel algorithmic framework, Curiosity Model Policy Optimization (CMPO), for enhanced robot control.
• To integrate curiosity-driven exploration with model-based approaches for improved learning efficiency.
• To reduce tracking errors and enhance generalization capabilities in robotic control tasks.

Main Methods:

• Developed a framework combining curiosity and model-based approaches (CMPO).
• Introduced a metric for judging positive and negative curiosity.
• Employed constrained optimization to update the curiosity ratio for efficient agent training.
• Defined a novelty distance buffer ratio to mitigate environment-model bias.
• Simulated CMPO against traditional controllers and baseline MBRL algorithms in a non-linear reward robotic environment.

Main Results:

• CMPO demonstrated superior tracking performance compared to traditional and baseline MBRL algorithms.
• The proposed algorithm exhibited enhanced generalization capabilities in robotic control tasks.
• The curiosity-driven approach and bias reduction techniques improved learning efficiency and performance.

Conclusions:

• CMPO offers a significant advancement in robot control, particularly in uncertain and saturated conditions.
• The integration of curiosity and model-based learning provides a robust framework for complex control problems.
• The developed methods for curiosity assessment and bias reduction are effective in improving agent performance and generalization.