Reinforcement learning based temperature control of a fermentation bioreactor for ethanol production

Area of Science:

• Biotechnology
• Chemical Engineering
• Artificial Intelligence

Background:

• Ethanol production is a key industrial bioprocess for energy generation.
• Advanced control systems for bioreactors often struggle with model-process mismatch.
• Reinforcement Learning (RL) offers a data-driven approach to learn control policies directly from the environment.

Purpose of the Study:

• To implement and evaluate the twin delayed deep deterministic policy gradient (TD3) algorithm for controlling ethanol bioreactor temperature.
• To compare the performance of TD3 against the deep deterministic policy gradient (DDPG) algorithm.
• To assess control strategies under both unconstrained and constrained conditions.

Main Methods:

• The study employed the TD3 algorithm, a type of RL, to control the reactor temperature in a nonlinear bioreactor model.
• Various reward functions, including integral absolute error (IAE), were tested with the TD3 algorithm.
• Performance was evaluated using mean squared error (MSE) and compared against the DDPG algorithm.

Main Results:

• The TD3 controller with IAE reward achieved a lower MSE of 0.22 in unconstrained control, compared to DDPG's 0.29.
• In constrained control, TD3 yielded an MSE of 0.38, outperforming DDPG's 0.48.
• The TD3 agent demonstrated effective disturbance rejection for flow rate and temperature variations.

Conclusions:

• The TD3 algorithm provides a robust and effective solution for bioreactor temperature control in ethanol production.
• TD3-based controllers offer superior performance over DDPG, particularly in minimizing errors and adapting to process changes.
• This RL approach enhances process stability and efficiency in industrial bioprocesses.