Minimum-Cost State-Flipped Control for Reachability of Boolean Control Networks Using Reinforcement Learning

Area of Science:

• Control Theory
• Artificial Intelligence
• Computer Science

Background:

• Boolean Control Networks (BCNs) are widely used for modeling complex systems.
• Controlling BCNs to achieve desired states efficiently is a significant challenge.
• Existing methods often struggle with large-scale networks and optimizing control costs.

Purpose of the Study:

• To propose model-free reinforcement learning methods for minimum-cost state-flipped control in BCNs.
• To determine the smallest flip set (flipping kernel) ensuring state reachability.
• To derive optimal policies minimizing flipping actions for reachability.

Main Methods:

• Utilizing Q-learning for reachability determination and optimal policy derivation.
• Incorporating transfer learning and specialized initial states to expedite Q-learning convergence.
• Developing a BCN-characteristics-based reward scheme for optimal control.
• Employing small memory Q-learning for large-scale BCNs, with an upper bound on memory usage.
• Introducing adaptive variable rewards to accelerate convergence in large-scale BCNs.

Main Results:

• Demonstrated the effectiveness of Q-learning in finding the flipping kernel and optimal control policies.
• Validated the proposed methods on both small- and large-scale BCNs.
• Showcased reduced memory usage and expedited convergence for large-scale networks.
• Proved the optimality of the BCN-characteristics-based reward scheme.

Conclusions:

• The proposed model-free reinforcement learning methods offer an effective solution for minimum-cost state-flipped control in BCNs.
• The techniques enhance reachability analysis and policy optimization, particularly for large-scale systems.
• The study contributes novel approaches to address computational challenges in BCN control.