Gr-GDHP: A New Architecture for Globalized Dual Heuristic Dynamic Programming.

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Machine Learning

Background:

• Traditional adaptive dynamic programming methods often rely on fixed or predefined reinforcement signals.
• Existing approaches may lack the flexibility for online adjustment of control parameters.
• Complex control tasks require sophisticated learning mechanisms for optimal performance.

Purpose of the Study:

• To introduce a novel Goal Representation Globalized Dual Heuristic Dynamic Programming (Gr-GDHP) method.
• To integrate a goal neural network for generating adaptive internal reinforcement signals.
• To simplify the critic network's learning process by utilizing signal derivatives.

Main Methods:

• The proposed Gr-GDHP method incorporates a goal neural network into the GDHP framework.
• Internal reinforcement signals and their derivatives are generated online.
• These signals directly influence the critic network's objective function.

Main Results:

• The internal reinforcement signal and its derivatives adapt online, offering greater flexibility.
• The critic network's learning process is simplified due to direct contribution of derivatives.
• Simulations demonstrate superior learning and control performance of Gr-GDHP over GDHP and other adaptive dynamic programming designs.
• Validation on a ball-and-beam balancing system confirms the method's effectiveness.

Conclusions:

• The Gr-GDHP method provides an effective approach for adaptive dynamic programming.
• Online adjustment of reinforcement signals enhances control and learning capabilities.
• The proposed method offers a simplified and improved learning process for control systems.