FG-BERT: a generalized and self-supervised functional group-based molecular representation learning framework for...

Area of Science:

• Computational chemistry and cheminformatics
• Artificial intelligence in drug discovery
• Deep learning for molecular modeling

Background:

• Accurate molecular property prediction is crucial for designing novel bioactive molecules and functional materials.
• Existing methods often require extensive feature engineering or lack interpretability.
• Deep learning offers potential for learning complex molecular representations.

Purpose of the Study:

• To introduce FG-BERT, a novel self-supervised deep learning framework for molecular property prediction.
• To enable learning of meaningful molecular representations directly from functional groups.
• To provide a highly performant and interpretable model for molecular design tasks.

Main Methods:

• Developed a self-supervised deep learning framework named Functional Group Bidirectional Encoder Representations from Transformers (FG-BERT).
• Pretrained FG-BERT on approximately 1.45 million unlabeled drug-like molecules.
• Fine-tuned the pretrained FG-BERT for various molecular property prediction tasks.
• Utilized attention mechanisms within FG-BERT for enhanced interpretability.

Main Results:

• FG-BERT demonstrated superior performance compared to state-of-the-art machine learning and deep learning methods across 44 benchmark datasets.
• The model achieved high accuracy in predicting properties related to physical chemistry, biophysics, and physiology.
• Attention mechanisms highlighted critical functional group features relevant to target properties, enhancing model interpretability.

Conclusions:

• FG-BERT provides an out-of-the-box framework for developing state-of-the-art models in molecular discovery, particularly for drug development.
• The framework eliminates the need for artificial feature engineering, simplifying the prediction process.
• FG-BERT offers excellent interpretability, aiding in understanding the basis of property predictions.