Adaptive learning in tracking control based on the dual critic network design.

Area of Science:

• Control Systems Engineering
• Machine Learning
• Artificial Intelligence

Background:

• Traditional adaptive dynamic programming often requires manual crafting of reinforcement signals.
• Existing methods may lack robust internal goal representations for adaptive learning.
• Optimizing system learning and performance in dynamic environments remains a challenge.

Purpose of the Study:

• To present a new adaptive dynamic programming approach integrating a reference network for internal goal representation.
• To introduce a dual critic network design for improved value function approximation.
• To offer an automated alternative to manual reinforcement signal generation.

Main Methods:

• Developed a dual critic network by integrating a reference network atop the critic network.
• Utilized an adaptively generated internal goal signal as a reinforcement signal.
• Employed online action-dependent heuristic dynamic programming (ADHDP) framework.
• Conducted Lyapunov stability analysis for theoretical validation.

Main Results:

• The dual critic network effectively approximates the value function using internal goal signals.
• Demonstrated real-time simulation capabilities on a virtual reality platform under disturbances.
• Achieved improved tracking control performance on two benchmarks compared to typical ADHDP.
• Validated the approach's effectiveness and stability through theoretical analysis and simulations.

Conclusions:

• The proposed adaptive dynamic programming approach with a dual critic network enhances learning and optimization.
• The integrated reference network provides an effective, automatically adjusted internal goal signal.
• The method offers a promising alternative for reinforcement signal design in control systems.
• Simulation results confirm superior performance and stability over existing methods.