Reinforcement Learning for Improving Chemical Reaction Performance.

Area of Science:

• Computational chemistry
• Machine learning in chemistry
• Drug discovery and development

Background:

• Deep learning (DL) methods are underutilized in chemical reaction prediction and generation.
• Chemical reactions are complex, involving multiple molecules and bond changes.
• Optimizing yield and selectivity in reaction discovery is crucial but challenging.

Purpose of the Study:

• Introduce RE-EXPLORE, a novel approach integrating deep reinforcement learning (RL) with deep generative models for chemical reaction discovery.
• Address the limitations of standard RL methods in exploring chemical space for novel reactants and catalysts.
• Enhance the discovery of high-yielding substrates and enantioselective catalysts.

Main Methods:

• Utilized a Recurrent Neural Network (RNN)-based deep generative model pretrained on large chemical databases (ChEMBL, ZINC, COCONUT).
• Integrated the generative model with deep reinforcement learning (RL) and a pretrained regressor for yield/selectivity estimation.
• Engineered a reward function incorporating a Tanimoto-based uniqueness factor to promote exploration and prevent repetitive molecule generation.
• Incorporated user-defined core fragments to guide the learning of specific reaction types.

Main Results:

• RE-EXPLORE successfully navigated chemical reaction space, identifying practically meaningful regions.
• The approach demonstrated notable improvements across three distinct reaction types.
• Identified high-yielding substrates and highly enantioselective chiral catalysts.
• The engineered reward function enhanced exploration and led to larger returns compared to standard RL.

Conclusions:

• RE-EXPLORE offers a powerful RL-based framework for expediting chemical reaction discovery.
• The method aids in the synthesis planning of important compounds, including pharmaceuticals.
• This approach has the potential to significantly advance the field of computational chemistry and drug development.