Research on intelligent regulation of layered water injection based on reinforcement learning SAC algorithm

Area of Science:

• Petroleum Engineering
• Artificial Intelligence
• Control Systems

Background:

• Water injection is crucial for maintaining oil reservoir pressure and ensuring stable oilfield production.
• Current control methods for layered water injection may lack the precision needed for optimal efficiency.
• Intelligent control offers potential for enhanced performance in oilfield operations.

Purpose of the Study:

• To develop and evaluate an intelligent control method for layered water injection columns using reinforcement learning.
• To construct a precise layered water injection model and algorithm for accurate flow control.
• To compare the performance of the proposed method against existing reinforcement learning algorithms.

Main Methods:

• A water injection column flow model was established, integrating device structure, pressure loss, and flow characteristics.
• An intelligent control system was designed using the Soft Actor-Critic (SAC) algorithm.
• A simulation platform using PyTorch was developed to train and test SAC, Proximal Policy Optimization (PPO), and Deep Deterministic Policy Gradient (DDPG) algorithms.

Main Results:

• The SAC algorithm achieved an average regulation error of 5% under a 5% threshold, outperforming PPO (37%).
• SAC demonstrated a regulation qualification rate of 81%, significantly higher than PPO (45%) and DDPG (60%).
• SAC required fewer adjustment steps (42% less than PPO, 28% less than DDPG) and showed greater stability and adaptability.

Conclusions:

• The SAC algorithm provides superior regulation accuracy, qualification rate, and response speed for intelligent layered water injection.
• This intelligent control method offers significant theoretical support and practical value for improving oilfield development efficiency.
• The developed code is available online to facilitate further research and application.