Molecular property prediction using pretrained-BERT and Bayesian active learning: a data-efficient approach to drug...

Area of Science:

• Computational chemistry
• Machine learning in drug discovery
• Bioinformatics

Background:

• Prioritizing compounds for testing is crucial in drug discovery.
• Active learning typically uses only labeled data, ignoring unlabeled molecular data.
• This limits predictive performance and molecule selection efficiency.

Purpose of the Study:

• To improve active learning in drug discovery by integrating pretrained transformer models.
• To address the limitation of fully supervised active learning by leveraging unlabeled molecular data.
• To enhance molecule selection and predictive performance in compound prioritization.

Main Methods:

• Integrated a transformer-based BERT model, pretrained on 1.26 million compounds, into an active learning pipeline.
• Separated representation learning from uncertainty estimation using pretrained molecular representations.
• Conducted experiments on Tox21 and ClinTox datasets.

Main Results:

• Achieved equivalent toxic compound identification with 50% fewer iterations compared to conventional active learning.
• Demonstrated that pretrained BERT representations create a structured embedding space for reliable uncertainty estimation with limited labeled data.
• Showcased improved model performance and acquisition efficiency in drug discovery.

Conclusions:

• High-quality molecular representations are fundamental to active learning success in drug discovery.
• The proposed framework integrates pretrained transformer models with Bayesian active learning for efficient screening.
• This approach provides a scalable foundation for optimizing compound prioritization in pharmaceutical research.