Materials discovery with extreme properties via reinforcement learning-guided combinatorial chemistry

Area of Science:

• Computational chemistry
• Materials science
• Machine learning

Background:

• Materials discovery aims to find superior materials, often requiring extrapolation beyond existing data.
• Traditional machine learning models struggle with extrapolation due to their reliance on learning data probability distributions.

Purpose of the Study:

• To develop a novel approach for discovering superior molecules beyond the capabilities of current machine learning models.
• To demonstrate the efficacy of reinforcement learning-guided combinatorial chemistry in identifying molecules with extreme target properties.

Main Methods:

• Development of a rule-based molecular designer utilizing reinforcement learning (RL).
• The RL model employs a trained policy to select molecular fragments for constructing target molecules.
• The system is designed to generate all possible molecular structures from fragment combinations, enabling the discovery of unknown molecules.

Main Results:

• The RL-guided model significantly outperformed probability distribution-learning models in discovering molecules with extreme target properties.
• In an experiment targeting seven extreme properties, the model identified 1315 molecules hitting all targets and 7629 hitting five targets out of 100,000 trials.
• All generated molecules were confirmed to be 100% chemically valid.

Conclusions:

• Reinforcement learning-guided combinatorial chemistry is a more effective strategy for discovering superior compounds compared to probability distribution-learning models.
• The developed method shows practical utility in real-world applications, including the discovery of protein docking molecules and HIV inhibitors.