Benchmarking Large Language Models for Polymer Property Predictions.

Area of Science:

• Polymer Science
• Materials Informatics
• Artificial Intelligence

Background:

• Machine learning and AI accelerate polymer property prediction and generative design.
• Large Language Models (LLMs) offer new avenues in polymer informatics.
• LLM-based methods simplify training by using natural language inputs, bypassing traditional needs for large labeled datasets and complex feature engineering.

Purpose of the Study:

• To fine-tune general-purpose LLMs (Llama-3-8B and GPT-3.5) for predicting key polymer thermal properties (Tg, Tm, Td).
• To benchmark LLM performance against traditional fingerprinting-based methods (Polymer Genome, polyGNN, polyBERT) using single-task (ST) and multi-task (MT) learning.
• To analyze LLM molecular embeddings and understand their limitations in capturing polymer chemo-structural information.

Main Methods:

• Fine-tuning of open-source Llama-3-8B and commercial GPT-3.5 on a dataset of 11,740 polymer entries.
• Parameter-efficient fine-tuning and hyperparameter optimization.
• Benchmarking against Polymer Genome, polyGNN, and polyBERT under ST and MT learning frameworks.

Main Results:

• LLM-based polymer informatics methods approach, but generally underperform, traditional methods in predictive accuracy and computational efficiency.
• Fine-tuned Llama-3 consistently outperformed GPT-3.5.
• Single-task learning was more effective than multi-task learning for LLMs in this study.
• LLMs demonstrated limitations in capturing nuanced chemo-structural information compared to traditional domain-specific methods.

Conclusions:

• General-purpose LLMs face challenges in polymer informatics due to difficulties in capturing complex molecular structures compared to specialized methods.
• Open-source LLMs like Llama-3 offer flexibility and better performance than commercial counterparts in this context.
• Findings guide LLM selection and highlight the interplay between molecular embeddings and NLP in materials science.