Deep Reinforcement Learning-Based Self-Optimization of Flow Chemistry

Area of Science:

• Chemical Engineering
• Machine Learning
• Process Optimization

Background:

• Flow chemistry offers cost-effective and sustainable manufacturing but faces challenges in process development due to labor-intensive methods.
• Optimizing flow chemistry processes is crucial for efficient synthesis of key compounds like pharmaceuticals.
• Machine learning integration can mitigate experimental burdens and improve process efficiency.

Purpose of the Study:

• To demonstrate deep reinforcement learning (DRL) as an effective self-optimization strategy for imine synthesis in flow.
• To develop and evaluate a deep deterministic policy gradient (DDPG) agent for optimizing flow reactor conditions.
• To compare the performance of DRL against traditional optimization methods.

Main Methods:

• A deep deterministic policy gradient (DDPG) agent was designed to learn optimal operating conditions through interaction with a flow reactor environment.
• A mathematical model of the reactor was developed using experimental data to train the DDPG agent.
• Novel adaptive dynamic hyperparameter tuning was implemented to enhance DRL training performance, alongside comparisons with Bayesian optimization and trial-and-error.
• The DRL strategy was benchmarked against gradient-free methods (SnobFit, Nelder-Mead).

Main Results:

• The DDPG agent demonstrated superior performance in imine synthesis optimization compared to Nelder-Mead and SnobFit.
• The DRL approach reduced the number of required experiments by approximately 50% compared to Nelder-Mead and 75% compared to SnobFit.
• The DDPG agent showed better tracking of the global solution, indicating enhanced optimization capabilities.

Conclusions:

• Deep reinforcement learning provides a robust, efficient, and sustainable method for optimizing flow chemistry processes.
• This data-driven approach significantly reduces experimental workload and enhances process efficiency.
• The findings encourage broader integration of machine learning in chemical process design and operation.