Transmol: repurposing a language model for molecular generation.

Area of Science:

• Artificial Intelligence
• Computational Chemistry
• Drug Design

Background:

• Deep learning, particularly convolutional neural networks, has advanced various fields.
• Machine learning is increasingly applied in life sciences, chemistry, and drug design.
• Recent years have seen a rise in machine learning methods for generating diverse molecular libraries.

Purpose of the Study:

• To apply attention mechanisms, specifically transformer architectures, to the problem of de novo molecular generation.
• To evaluate the effectiveness of the adapted transformer model (Transmol) for generating molecular libraries.
• To introduce novel generation modes and validation strategies for machine learning-based molecular design.

Main Methods:

• Employed a variant of transformer architecture, originally developed for natural language processing, for molecular generation.
• Implemented a one-seed approach for input-guided generation of focused molecular libraries.
• Developed a two-seed approach for diversity-driven generation, exploring under-explored chemical spaces.
• Validated the one-seed approach by recreating known ligands for the vitamin D receptor.

Main Results:

• The adapted Transmol model successfully generates molecular libraries.
• The model achieved statistically significant improvements in key metrics on the MOSES benchmark.
• The one-seed and two-seed approaches enable targeted and exploratory molecular generation, respectively.
• The validation workflow confirmed the model's ability to generate relevant ligands.

Conclusions:

• Transformer-based models are effective for de novo molecular generation.
• The Transmol model offers flexible generation modes for chemical library design.
• The developed methods and tools advance machine learning applications in chemistry and drug discovery.
• The Transmol algorithm is integrated into the cheML.io web database for community access.