Reinforcement Learning for H∞ Optimal Control of Unknown Continuous-Time Linear Systems

Area of Science:

• Control Theory
• Machine Learning
• System Identification

Background:

• Designing optimal control for systems with unknown dynamics and disturbances is a significant challenge.
• Existing reinforcement learning methods for H-infinity optimal control often require difficult-to-monitor persistence of excitation (PE) conditions or large data storage.
• These limitations hinder practical online implementation of advanced control strategies.

Purpose of the Study:

• To develop novel reinforcement learning algorithms for H-infinity optimal control of continuous-time linear systems with unknown dynamics.
• To overcome the limitations of existing methods by removing the need for persistence of excitation or extensive data storage.
• To introduce an online-verifiable initial excitation condition for guaranteed algorithm convergence.

Main Methods:

• Development of initial excitation-based reinforcement learning algorithms.
• Analysis of algorithm properties to prove convergence under the initial excitation condition.
• Numerical simulations to validate the performance and correctness of the proposed algorithms.

Main Results:

• The proposed initial excitation-based reinforcement learning algorithms converge to the optimal control policy.
• The algorithms function effectively under an online-verifiable initial excitation condition.
• Numerical analysis confirms the practical applicability and correctness of the developed methods.

Conclusions:

• Initial excitation-based reinforcement learning offers a viable solution for H-infinity optimal control in systems with unknown dynamics.
• The new approach simplifies practical implementation by replacing stringent prior conditions with an easily verifiable initial excitation.
• This work advances the field of adaptive control by enabling robust and efficient learning of optimal control policies.