Learning a chemistry-aware latent space for molecular encoding and generation with a large-scale transformer...

Area of Science:

• Computational Chemistry
• Cheminformatics
• Machine Learning

Background:

• Exploring vast chemical spaces for molecule optimization is challenging due to data limitations and chemical space complexity.
• Existing methods for mapping chemical space to latent embeddings often fail in reconstruction accuracy, latent space structure, and chemical similarity representation.
• Lack of readily available code and large-scale trained models hinders practical application of generative chemistry models.

Purpose of the Study:

• To develop a large-scale, pre-trained Variational Autoencoder utilizing the Transformer architecture for molecular representation.
• To create a chemistry-aware, structured latent space enabling efficient exploration and interpolation of molecular properties.
• To provide a reliable and user-friendly tool for drug design and molecular optimization tasks.

Main Methods:

• Implementation of a Transformer-based Variational Autoencoder for converting small organic molecules into fixed-size continuous embeddings and vice versa.
• Development of a novel training objective with a loss term that enforces Tanimoto similarity between molecular fingerprints and embedding distances.
• Training on a large-scale dataset to ensure robust molecular representation and generation capabilities.

Main Results:

• Achieved a 97% reconstruction rate and 100% validity rate for generated molecules.
• Demonstrated a well-structured and smooth latent space where distances accurately reflect chemical similarities.
• The model showed solid performance in molecular optimization tasks when compared to other generative models.
• Generated a high diversity of novel, valid molecules.

Conclusions:

• The developed model offers a reliable mapping between chemical space and a structured latent space, overcoming limitations of previous methods.
• The chemistry-aware latent space facilitates seamless exploration and interpolation for molecular design.
• The open-source release of the model and code promotes practical application in drug discovery and development.