SINDy-RL for interpretable and efficient model-based reinforcement learning

Area of Science:

• Control Theory
• Machine Learning
• Fluid Dynamics

Background:

• Deep reinforcement learning (DRL) excels at complex control but demands extensive data and yields black-box policies.
• Sparse dictionary learning methods like SINDy offer efficient, interpretable models, particularly in low-data scenarios.

Purpose of the Study:

• To introduce SINDy-RL, a unified framework integrating SINDy and DRL.
• To develop efficient, interpretable, and trustworthy data-driven models for dynamics, rewards, and control policies.
• To address the data inefficiency and interpretability limitations of conventional DRL.

Main Methods:

• Integration of sparse identification of nonlinear dynamics (SINDy) with deep reinforcement learning (DRL).
• Development of a unifying framework (SINDy-RL) for learning dynamics, reward functions, and control policies.
• Application to benchmark control tasks and flow control problems, including gust mitigation on an airfoil.

Main Results:

• SINDy-RL achieves performance comparable to state-of-the-art DRL algorithms.
• The framework requires significantly fewer environmental interactions for training compared to traditional DRL.
• The resulting control policy is orders of magnitude smaller and more interpretable than DRL-derived policies.

Conclusions:

• SINDy-RL offers a more data-efficient and interpretable alternative to standard DRL for control tasks.
• The framework provides trustworthy and computationally efficient models suitable for various applications, including embedded systems.
• This approach enhances the practical applicability of reinforcement learning in complex dynamic environments.