Knowledge-based BERT: a method to extract molecular features like computational chemists.

Area of Science:

• Computational chemistry
• Cheminformatics
• Machine learning in drug discovery

Background:

• Machine learning models are crucial for early-stage drug discovery, but SMILES-based methods are less popular due to feature extraction challenges.
• Descriptor- and graph-based methods are mainstream, yet SMILES-based approaches offer direct feature extraction without expert knowledge.

Purpose of the Study:

• To demonstrate the potential of pre-training for improving molecular property predictions.
• To introduce K-BERT, a novel pre-training method for extracting chemical information from SMILES.
• To evaluate K-BERT's performance against established methods.

Main Methods:

• Developed K-BERT utilizing three pre-training tasks: atom feature prediction, molecular feature prediction, and contrastive learning.
• Applied K-BERT to 15 pharmaceutical datasets for molecular property prediction.
• Generated K-BERT-FP fingerprints and compared their predictive power to MACCS.

Main Results:

• K-BERT significantly outperformed established descriptor-based (XGBoost) and graph-based (Attentive FP, HRGCN+) models.
• Contrastive learning enabled K-BERT to interpret non-canonical SMILES.
• K-BERT-FP demonstrated comparable predictive power to MACCS and captured molecular size and chirality information.

Conclusions:

• K-BERT shows great potential for practical applications in molecular property prediction for drug discovery.
• Pre-training is a powerful strategy to enhance SMILES-based molecular property prediction.
• K-BERT offers a novel and effective approach for chemical information extraction from SMILES.