Solving the Zero-Sum Control Problem for Tidal Turbine System: An Online Reinforcement Learning Approach

Area of Science:

• Control Systems Engineering
• Renewable Energy Systems
• Game Theory

Background:

• Tidal turbine control is complex due to unknown dynamics and system uncertainties.
• Markov jump linear models are suitable for systems with changing operational modes.
• Finding Nash equilibrium is crucial for optimal control in multi-agent or game-theoretic scenarios.

Purpose of the Study:

• To develop a novel algorithm for computing optimal control policies in tidal turbine systems.
• To address two-player zero-sum games and Nash equilibrium problems under uncertainty.
• To achieve optimal control without prior knowledge of system dynamics.

Main Methods:

• Modeling the tidal turbine system as a continuous-time Markov jump linear system.
• Employing a subsystem transformation to decouple system modes.
• Utilizing a completely mode-free integral reinforcement learning (CMFIRL) algorithm to solve game-coupled algebraic Riccati equations.
• Implementing an iterative learning approach updating control and disturbance policies simultaneously with an exploration signal.

Main Results:

• The proposed CMFIRL algorithm successfully computes optimal control policy pairs for the tidal turbine system.
• The algorithm achieves Nash equilibrium without requiring system dynamics information.
• Rigorous proof of convergence for the CMFIRL iteration algorithm is provided.
• Simulation results demonstrate the effectiveness and applicability of the control design.

Conclusions:

• The CMFIRL-based iteration algorithm offers a robust solution for optimal control of tidal turbines with unknown dynamics.
• This method effectively solves complex game-theoretic problems in renewable energy systems.
• The approach is validated through simulations, showing practical applicability.