Dual Embedding: A Fine-Tuned Language Model Approach for Accurate Polymer Glass Transition Temperature Prediction.

Area of Science:

• Polymer Science
• Materials Informatics
• Machine Learning

Background:

• Polymer informatics leverages language models (e.g., BERT) for property prediction.
• Current methods using embeddings primarily capture chemical structure, not physicochemical properties like glass transition temperature (Tg).
• Accurate prediction of Tg remains a significant challenge in polymer science.

Purpose of the Study:

• To develop an enhanced dual-embedding framework for improved Tg prediction.
• To create embeddings where vector similarity directly correlates with Tg values.
• To advance polymer informatics beyond structure-centric approaches.

Main Methods:

• Introduced a dual-embedding framework combining standard BERT embeddings with a fine-tuned BERT model.
• The fine-tuned model was explicitly trained to ensure vector similarity reflects proximity in Tg values.
• Evaluated the framework on four diverse datasets, including heterogeneous polymers, homopolymers, and polyimides, comparing against 25 machine learning baselines.

Main Results:

• The dual-embedding approach significantly improved Tg prediction accuracy compared to standard BERT embeddings.
• Achieved up to a 20% reduction in root-mean-square error (RMSE) for Tg prediction.
• Outperformed alternative methods, including graph-based and descriptor-based approaches, across multiple benchmarks.

Conclusions:

• Embedding molecular properties directly into representations enhances predictive accuracy in polymer informatics.
• The dual-embedding framework offers a more effective approach to predicting polymer properties like Tg.
• This work signifies a move towards property-aware representations in polymer data science.