A Theoretical Foundation of Goal Representation Heuristic Dynamic Programming.

Area of Science:

• Control Theory
• Dynamic Programming
• Reinforcement Learning

Background:

• Goal Representation Heuristic Dynamic Programming (GrHDP) is a recent control design.
• Its effectiveness has been shown in case studies and industrial applications.
• Prior research lacks theoretical analysis of the internal reinforcement signal.

Purpose of the Study:

• To provide a theoretical analysis of the GrHDP control design.
• To investigate the properties of the internal reinforcement signal in GrHDP.
• To establish the theoretical foundation for GrHDP's performance improvements.

Main Methods:

• Theoretical analysis under specific conditions.
• Mathematical proofs for signal boundedness and convergence.
• Simulation studies for validation.

Main Results:

• The internal reinforcement signal is proven to be bounded.
• The performance index converges monotonically to its optimal value.
• The existence of admissible control for GrHDP is demonstrated.

Conclusions:

• This study presents the first theoretical foundation for GrHDP's internal reinforcement signal.
• The internal reinforcement signal provides effective information for control performance enhancement.
• The theoretical findings are validated through extensive simulations.