Enhancing Molecular Property Prediction through Task-Oriented Transfer Learning: Integrating Universal Structural...

Area of Science:

• Computational chemistry
• Drug discovery
• Machine learning

Background:

• Accurate molecular property prediction is vital for drug discovery.
• Deep learning methods are challenged by limited labeled data.
• Existing self-supervised pretraining often overlooks crucial domain-specific knowledge.

Purpose of the Study:

• To introduce a novel dual-level pretraining framework, Task-Oriented Multilevel Learning based on BERT (TOML-BERT).
• To address the limitations of current pretraining methods by incorporating both molecular structure and domain knowledge.
• To enhance the performance of deep learning models in molecular property prediction.

Main Methods:

• Developed TOML-BERT, a dual-level pretraining framework utilizing BERT architecture.
• Integrated molecular structural patterns and domain-specific knowledge into the pretraining process.
• Employed massive pseudo-labeled data for knowledge extraction and contextual information mining within molecular structures.

Main Results:

• Achieved state-of-the-art prediction performance across 10 diverse pharmaceutical datasets.
• Demonstrated significant positive transfer through the complementary contributions of the dual-level pretraining components.
• Showcased the ability to mine contextual information and extract domain knowledge effectively.

Conclusions:

• TOML-BERT significantly advances molecular property prediction in drug discovery.
• The dual-level pretraining effectively learns task-related molecular representations.
• Combining multiple pretraining tasks holds great potential for extracting task-oriented knowledge.