Transferring a Molecular Foundation Model for Polymer Property Predictions.

Area of Science:

• Computational chemistry
• Materials science
• Polymer science

Background:

• Transformer-based large language models (LLMs) show promise for accelerating design optimization in fields like drug development and material discovery.
• Self-supervised pretraining of LLMs necessitates large datasets, which are scarce in specialized domains such as polymer science.
• Current methods for polymers involve data augmentation, increasing computational expenses.

Purpose of the Study:

• To investigate the efficacy of transfer learning for polymer property prediction using LLMs.
• To address data scarcity in polymer science by leveraging readily available small molecule datasets.
• To compare the performance of transfer learning against traditional data augmentation techniques.

Main Methods:

• Utilized transformer-based LLMs pretrained on extensive small molecule datasets.
• Fine-tuned the pretrained models on specific polymer property datasets.
• Evaluated model accuracy on benchmark prediction tasks for polymers.
• Compared results against models trained solely on augmented polymer data.

Main Results:

• LLMs pretrained on small molecules and fine-tuned on polymers achieved accuracy comparable to models trained on augmented polymer data.
• Transfer learning proved effective in overcoming data scarcity in polymer science.
• This approach offers a computationally efficient alternative to data augmentation.

Conclusions:

• Transfer learning with LLMs offers a viable and efficient strategy for polymer property prediction.
• Pretraining on small molecules provides a robust foundation for models applied to polymer science.
• This method reduces the need for extensive data augmentation, saving computational resources.