Meta-learning for transformer-based prediction of potent compounds

Area of Science:

• Drug Discovery
• Machine Learning
• Computational Chemistry

Background:

• Limited training data is a key challenge for machine learning (ML) in drug discovery, particularly for compound design and activity prediction.
• Deep learning models often require substantial data, hindering their application in low-data scenarios.
• Meta-learning offers a potential solution by enabling learning in low-data regimes through model output combination and meta-data utilization.

Purpose of the Study:

• To explore the efficacy of meta-learning for predicting potent compounds using generative transformer models in drug discovery.
• To assess the performance of meta-learning models compared to standard transformers under varying fine-tuning data conditions.
• To evaluate the potency and selectivity of compounds generated by meta-learning models.

Main Methods:

• Developed and applied meta-learning strategies with transformer models for generative compound design.
• Trained models to predict highly potent compounds from weakly potent templates across different activity classes.
• Compared meta-learning model performance against conventional transformer models using varying amounts of fine-tuning data.

Main Results:

• Meta-learning consistently yielded statistically significant improvements in predictive model performance.
• Performance gains were particularly pronounced when fine-tuning data were limited.
• Compounds generated by meta-learning models exhibited higher potency and greater potency differences compared to those from other transformers.

Conclusions:

• Meta-learning demonstrates significant potential for low-data compound design in drug discovery.
• The approach effectively enhances the prediction of potent compounds, even with scarce training data.
• Meta-learning facilitates the generation of compounds with improved potency and selectivity profiles.