Boundary Optimal Control for Parabolic Distributed Parameter Systems With Value Iteration.

Area of Science:

• Control Theory
• Applied Mathematics
• Machine Learning

Background:

• Parabolic distributed parameter systems present significant control challenges.
• Traditional optimal control methods often rely on complex mathematical theories.
• Developing efficient algorithms for these systems is crucial for various applications.

Purpose of the Study:

• To develop a reinforcement learning-based boundary optimal control algorithm for parabolic distributed parameter systems.
• To avoid complex mathematical frameworks like semigroups and operator theories.
• To provide an effective computational method for solving infinite-time horizon optimal control problems.

Main Methods:

• Derivation of a spatial Riccati-like equation and an integral optimal controller using variational methods.
• Proposal of a value iteration algorithm based on the Bellman optimality principle to solve the Riccati-like equation and derive the optimal control law.
• Implementation of the value iteration algorithm using an approximation scheme with radial basis functions and weighted residuals.

Main Results:

• The convergence of the value iteration algorithm is mathematically proven.
• The proposed algorithm successfully obtains the solution to the spatial Riccati-like equation and the optimal control law.
• Simulations on a diffusion-reaction process confirm the method's effectiveness.

Conclusions:

• The developed reinforcement learning algorithm offers a computationally tractable approach for boundary optimal control of parabolic systems.
• The method successfully bypasses the need for advanced operator theories.
• The approach demonstrates practical applicability and effectiveness in simulation studies.