Fuzzy Lyapunov Reinforcement Learning for Non Linear Systems

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Fuzzy Logic Systems

Background:

• Traditional control methods often require precise system models and struggle with uncertainties.
• Reinforcement learning (RL) offers adaptive control but ensuring stability, especially in fuzzy systems, remains a challenge.

Purpose of the Study:

• To develop a novel fuzzy reinforcement learning (RL) controller capable of generating stable control actions.
• To introduce Lyapunov constraining of fuzzy rule consequents within an RL framework for guaranteed stability.
• To create a model-free and desired-response-free controller for continuous state-action spaces.

Main Methods:

• Lyapunov constraining the consequent part of fuzzy linguistic rules within a fuzzy RL setup.
• Developing a linguistic RL controller that progressively learns a stable optimal policy.
• Applying the controller to benchmark Inverted Pendulum (IP) and Rotational/Translational Proof-Mass Actuator (RTAC) problems.

Main Results:

• The proposed controller successfully generated stable control actions in continuous state-action spaces.
• Demonstrated effectiveness in handling disturbances on benchmark control problems.
• Simulation results showed superior stability and viability compared to baseline fuzzy Q learning, Lyapunov Actor-Critic, and Lyapunov Markov game controllers.

Conclusions:

• The proposed fuzzy RL controller with Lyapunov constrained consequents is a viable and stable approach for complex control tasks.
• This method offers a robust, model-free solution for systems with uncertainties and disturbances.
• Represents a significant advancement in linguistic RL for achieving stable optimal policies.