Relative Entropy of Correct Proximal Policy Optimization Algorithms with Modified Penalty Factor in Complex...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Reinforcement Learning

Background:

• Traditional reinforcement learning algorithms face challenges with robustness and stationarity.
• Proximal Policy Optimization (PPO) is a widely used algorithm but can be improved.

Purpose of the Study:

• To propose a novel Correct Proximal Policy Optimization (CPPO) algorithm.
• To address the robustness and stationarity issues in traditional reinforcement learning algorithms.
• To enhance policy learning in complex environments.

Main Methods:

• Developed a strategy evaluation mechanism using policy distribution functions.
• Quantified state space using entropy and approximated real policy distributions.
• Utilized kernel function estimation and relative entropy to fit reward functions for complex problems.

Main Results:

• CPPO demonstrated superior effectiveness, faster convergence, and better performance than traditional PPO on classic test cases.
• Relative entropy measure effectively highlights performance differences.
• The algorithm efficiently leverages complex environmental information for policy learning.

Conclusions:

• The proposed CPPO algorithm offers significant improvements in reinforcement learning.
• The framework balances iteration steps, computational complexity, and convergence speed.
• Relative entropy serves as an effective performance measure in complex reinforcement learning scenarios.