Fuzzy-TD3: A prior-knowledge guided deterministic policy gradient for interpretable control

Area of Science:

• Artificial Intelligence
• Machine Learning
• Control Systems

Background:

• Deterministic policy gradient methods like TD3 offer high performance but lack interpretability.
• Existing fuzzy reinforcement learning methods provide limited interpretability within complex pipelines.
• Integrating rule-level prior knowledge into fuzzy actors within TD3 is a significant challenge.

Purpose of the Study:

• To develop an interpretable reinforcement learning algorithm, Fuzzy-TD3, addressing the limitations of current methods.
• To replace neural network actors in TD3 with a Takagi-Sugeno (T-S) fuzzy policy.
• To integrate prior knowledge at the rule level within the fuzzy actor-critic framework.

Main Methods:

• Proposed Fuzzy-TD3, a deterministic actor-critic algorithm utilizing a T-S fuzzy policy.
• Fixed consequents of fuzzy rules in linearized operating regions to an optimal feedback law.
• Learned remaining rule consequents from data and reparameterized antecedents for gradient descent training.

Main Results:

• Fuzzy-TD3 demonstrated improved sample efficiency and steady-state regulation on inverted-pendulum benchmarks.
• Maintained competitive transient and input-usage performance compared to neural actor-critic baselines.
• Achieved enhanced interpretability through a fuzzy policy structure.

Conclusions:

• Fuzzy-TD3 offers a practical and interpretable reinforcement learning framework by merging fuzzy theory and classical control.
• The proposed method provides a robust solution for data-driven control in industrial applications.
• This approach successfully integrates prior knowledge at the rule level, enhancing learning efficiency and interpretability.