Self-Supervised Molecular Pretraining Strategy for Low-Resource Reaction Prediction Scenarios

Area of Science:

• Computational Chemistry
• Machine Learning in Chemistry
• Chemical Reaction Prediction

Background:

• Low-resource reaction training samples pose a significant challenge for developing accurate chemical prediction models.
• Existing models often struggle with datasets lacking extensive reaction data.

Purpose of the Study:

• To develop a robust chemical platform for addressing small-scale reaction prediction problems.
• To enhance the predictive performance of Transformer models in low-resource scenarios.

Main Methods:

• Utilized a self-supervised pretraining strategy, Masked Sequence to Sequence (MASS), on approximately 1 billion molecules.
• Fine-tuned the Transformer model on small-scale reaction prediction tasks.
• Integrated MASS with a reaction transfer learning strategy to further boost predictive capabilities.

Main Results:

• Achieved average improved accuracies of 14.07% for Baeyer-Villiger reactions.
• Demonstrated average improved accuracies of 24.26% for Heck reactions.
• Showcased average improved accuracies of 40.31% for C-C bond formation reactions.
• Reported average improved accuracies of 57.69% for functional group interconversion reactions.

Conclusions:

• The combined MASS pretraining and transfer learning approach significantly enhances Transformer model performance in low-resource reaction prediction.
• This work represents a crucial advancement for developing predictive models with limited chemical reaction data.