Deep reinforcement learning for the direct optimization of gradient separations in liquid chromatography

Area of Science:

• Analytical Chemistry
• Separation Science
• Chromatography

Background:

• Reinforcement learning (RL) has shown success in complex tasks but is largely unexplored in separation sciences.
• Optimizing separation gradients in liquid chromatography is crucial for resolving complex mixtures.
• Traditional gradient optimization methods can be time-consuming and may not yield optimal results.

Purpose of the Study:

• To introduce proximal policy optimization (PPO), a type of RL, to liquid chromatography.
• To evaluate PPO's ability to optimize separation gradients based on single scouting runs.
• To investigate if PPO can improve separation resolution by learning tailored gradient programs.

Main Methods:

• PPO agents were trained to select linear and multi-segment (2-, 3-, 16-segment) gradient programs.
• Optimization was based on the outcome of an initial generic linear gradient.
• Agents were also trained using sequential experiments (2 or 3 runs) to further refine gradients.

Main Results:

• The PPO agent successfully improved separations by selecting optimized gradient programs (ϕ-programs).
• Multi-segment gradients and sequential experiments led to increased rewards (improved resolution), reaching an average of 0.918.
• PPO significantly outperformed random and standard gradient optimization methods, achieving higher average rewards.

Conclusions:

• Proximal policy optimization (PPO) demonstrates significant potential for automated gradient optimization in liquid chromatography.
• RL agents can learn to tailor separation conditions for specific mixtures, improving peak resolution.
• This study presents a promising new direction for advancing separation science through artificial intelligence.