Relaxed Optimal Control With Self-Learning Horizon for Discrete-Time Stochastic Dynamics

Area of Science:

• Control Theory
• Machine Learning
• Optimization

Background:

• Traditional optimal learning control methods often focus on either initialization or acceleration.
• Improving learning ability is key to advancing optimal control.
• Existing methods face challenges with initialization burden and slow optimization.

Purpose of the Study:

• To develop a novel optimal learning control algorithm integrating initialization and acceleration.
• To introduce a relaxed policy iteration (PI) algorithm with a self-learning horizon.
• To address limitations of traditional methods in stochastic optimal control.

Main Methods:

• Developed a novel relaxed policy iteration (PI) algorithm.
• Incorporated a self-learning horizon mechanism for policy evaluation.
• Analyzed algorithm convergence and system stability for relaxed optimal control.
• Applied the algorithm to unconventional problems including external noises and nonzero equilibrium.

Main Results:

• The self-learning horizon allows direct evaluation of inadmissible policies, reducing initialization burden.
• Inadmissible policies are rapidly optimized with fewer learning iterations.
• The relaxed PI algorithm effectively solves unconventional stochastic optimal control problems.
• Experimental results on nonlinear benchmarks confirm the effectiveness of the self-learning horizon.

Conclusions:

• The proposed relaxed PI algorithm with a self-learning horizon offers a superior approach to optimal learning control.
• The self-learning horizon mechanism significantly enhances the efficiency and applicability of optimal control algorithms.
• The method demonstrates strong performance in practical applications with complex dynamics.