Adaptive Observation-Based Efficient Reinforcement Learning for Uncertain Systems.

Area of Science:

• Control Systems Engineering
• Machine Learning
• Adaptive Systems

Background:

• Systems with uncertain drift dynamics pose significant control challenges.
• Traditional reinforcement learning (RL) often requires persistent excitation for convergence.
• Accurate state and parameter estimation is crucial for effective control.

Purpose of the Study:

• To develop an adaptive observation-based reinforcement learning (RL) approach for systems with uncertain drift dynamics.
• To design a concurrent learning adaptive extended observer (CL-AEO) for joint state and parameter estimation.
• To achieve optimal control policy approximation without the need for persistence of excitation (PE).

Main Methods:

• A novel two-time-scale concurrent learning adaptive extended observer (CL-AEO) was developed.
• The CL-AEO estimates system state and parameters without requiring state derivative calculations.
• An experience-based RL scheme was implemented using CL-AEO outputs for online policy approximation.

Main Results:

• The CL-AEO successfully estimates system states and parameters under uncertain dynamics.
• The developed RL approach demonstrated practical convergence of system states to the origin.
• The control policy approximated the ideal optimal policy without the persistence of excitation (PE) condition.

Conclusions:

• The proposed adaptive observation-based RL method is effective for systems with uncertain drift dynamics.
• The concurrent learning adaptive extended observer (CL-AEO) relaxes excitation conditions for parameter estimation.
• The methodology offers a superior approach for achieving optimal control in complex systems.