Dynamic, Unconstrained Optimization of Secreted Enzyme Production in Fed-Batch Fermentation Using Reinforcement...

Area of Science:

• Biotechnology
• Biochemical Engineering
• Artificial Intelligence

Background:

• Fed-batch fermentation is crucial for producing enzymes like glucoamylase but is sensitive to process variations and initial conditions.
• Traditional control methods struggle with unconstrained product maximization and process perturbations.
• Reinforcement learning (RL) offers a promising approach for dynamic process control in complex biological systems.

Purpose of the Study:

• To develop and evaluate a reinforcement learning (RL) agent for unconstrained product maximization in Aspergillus niger fed-batch fermentation.
• To compare the performance of different RL algorithms (PPO and SAC) and assess their robustness to process perturbations.
• To benchmark the RL controller against traditional model-free methods and evaluate its adaptability to new cell types.

Main Methods:

• A digital twin of Aspergillus niger fed-batch fermentation was created using the Monod model and literature parameters.
• Reinforcement learning agents (PPO and SAC) were trained on this digital environment, using state variables like run time, cell concentration, and enzyme activity.
• The RL controller's performance was compared to a Bayesian optimization-based model-free controller and tested against simulated process perturbations and new cell types.

Main Results:

• Soft Actor-Critic (SAC) demonstrated superior performance, reaching 95% of maximum quality within 2400 episodes, outperforming Proximal Policy Optimization (PPO).
• The RL controller maintained higher enzyme production than the traditional controller, even when faced with simulated process perturbations like faulty pumps or variations in cell growth.
• The in silico trained RL agent showed good performance on new cell types without updates and improved with retraining, indicating adaptability.

Conclusions:

• Reinforcement learning, particularly the SAC algorithm, provides a robust and effective method for optimizing fed-batch fermentation and maximizing enzyme production.
• RL controllers exhibit greater resilience to process variations and perturbations compared to traditional model-free controllers.
• In silico training followed by targeted experimental updates allows for the development of adaptable and high-performing RL agents for bioprocess control.