Difficulty in chirality recognition for Transformer architectures learning chemical structures from string...

Area of Science:

• Computational chemistry
• Machine learning in chemistry

Background:

• Representation learning for molecular descriptors is advancing rapidly.
• Natural language processing (NLP) models applied to SMILES strings are a key area.
• The interpretability of these NLP models in understanding chemical structures remains under-explored.

Purpose of the Study:

• To investigate how NLP models, specifically the Transformer, learn chemical structures from SMILES.
• To understand the relationship between model learning progress and chemical structure comprehension.
• To identify challenges in NLP model learning, particularly regarding stereochemistry.

Main Methods:

• Utilized a Transformer model, a representative NLP architecture.
• Trained the model on SMILES strings representing diverse molecules.
• Analyzed the learning progress concerning partial and overall molecular structures.
• Evaluated molecular property prediction accuracy at different training stages.
• Investigated the model's ability to learn chirality and handle enantiomers.

Main Results:

• The Transformer model rapidly learns partial molecular structures but requires prolonged training for holistic understanding.
• Molecular property prediction accuracy remained consistent throughout training, irrespective of learning stage.
• The model demonstrated significant challenges in learning chirality, often misinterpreting enantiomers, leading to performance stagnation.

Conclusions:

• NLP models like the Transformer exhibit distinct learning trajectories for different aspects of molecular structure.
• Extended training is crucial for NLP models to grasp complex chemical features such as chirality.
• These findings contribute to a deeper mechanistic understanding of NLP applications in cheminformatics.