Feedback stabilization of probabilistic finite state machines based on deep Q-network

Area of Science:

• Control Theory
• Machine Learning
• Artificial Intelligence

Background:

• Finite state machines (FSMs) are crucial mathematical models in manufacturing and healthcare.
• Traditional FSM analysis and design methods face challenges with convenience and computational complexity.
• Probabilistic finite state machines (PFSMs) present unique stabilization challenges.

Purpose of the Study:

• To address the limitations of traditional methods in analyzing and designing finite state machines.
• To develop an efficient algorithm for the stabilization of probabilistic finite state machines.
• To leverage deep learning techniques for solving complex control problems in FSMs.

Main Methods:

• Recalled preliminaries on Markov decision process, epsilon-greedy strategy, and deep Q-network (DQN).
• Derived a necessary and sufficient stabilizability condition for PFSMs.
• Transformed the PFSM feedback stabilization problem into an optimization problem solvable with DQN.

Main Results:

• Developed a novel algorithm using DQN to compute state feedback stabilizers for PFSMs.
• Demonstrated that DQN overcomes the limited capacity issues of traditional Q-learning.
• The proposed method efficiently handles high-dimensional and complex systems.

Conclusions:

• The DQN-based approach offers an efficient and scalable solution for PFSM stabilization.
• This method enhances the analysis and design capabilities for complex FSM-based systems.
• The findings open new avenues for applying deep reinforcement learning in control theory.