A systematic review of molecular representation learning foundation models

Area of Science:

• Computational Chemistry
• Machine Learning
• Drug Discovery

Background:

• Molecular representation learning (MRL) transforms molecular structures into numerical vectors for computational analysis.
• Foundation models offer new opportunities for MRL, improving generalizability and performance, especially with limited data.

Purpose of the Study:

• To review current molecular descriptors, datasets, and foundation models in MRL.
• To classify foundation models and analyze pretraining strategies for MRL.
• To discuss the application and interpretability of MRL foundation models in drug discovery.

Main Methods:

• Classification of foundation models into unimodal-based and multimodal-based categories.
• Evaluation of representative models, their advantages, and disadvantages.
• Systematic summary of four core pretraining strategies for MRL foundation models.

Main Results:

• Foundation models enhance MRL by improving generalizability and adaptability through pretraining and fine-tuning.
• Identified and evaluated unimodal and multimodal foundation models for MRL.
• Analyzed pretraining strategies and their impact on downstream tasks in drug discovery.

Conclusions:

• MRL foundation models are crucial for advancing computational drug discovery.
• Future directions include enhancing model interpretability and exploring novel pretraining techniques.
• The review provides a comprehensive overview of MRL foundation models for small molecules.