Reinforcement-learning-based output-feedback control of nonstrict nonlinear discrete-time systems with application to...

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Automotive Engineering

Background:

• Controlling nonlinear discrete-time systems with unknown disturbances is challenging.
• Existing methods often require specific system properties or assumptions.
• Adaptive-critic neural network (NN) controllers offer a promising alternative.

Purpose of the Study:

• To develop a novel reinforcement-learning-based output adaptive NN controller (adaptive-critic NN controller).
• To achieve desired tracking performance for nonlinear discrete-time systems under bounded, unknown disturbances.
• To validate the controller's effectiveness on a spark ignition (SI) engine.

Main Methods:

• The adaptive-critic NN controller integrates an observer, a critic NN, and two action NNs.
• Online weight adaptation uses a gradient-descent-based rule, avoiding iteration-based schemes.
• Lyapunov functions ensure the stability of tracking errors, weights, and observer estimates.

Main Results:

• The controller demonstrated a 25% reduction in cyclic dispersion for an SI engine.
• Oxides of nitrogen (NOx) emissions decreased by 30%, and unburned hydrocarbons by 16%.
• Overall NOx reduction reached over 80% compared to stoichiometric levels, with minimal fuel input change (<1%).

Conclusions:

• The developed adaptive-critic NN controller effectively manages nonlinear discrete-time systems with disturbances.
• The controller achieves significant emission reductions and performance improvements in SI engines.
• This approach bypasses limitations of traditional control methods, such as separation principles and persistency of excitation.