Accurate, interpretable predictions of materials properties within transformer language models

Area of Science:

• Materials Informatics
• Computational Materials Science
• Artificial Intelligence in Science

Background:

• Machine learning models in materials informatics achieve high accuracy but often function as "black boxes," lacking interpretability.
• The need for transparent and understandable models is crucial for advancing materials discovery and design.
• Current methods struggle to balance predictive power with clear explanations of model reasoning.

Purpose of the Study:

• To develop an interpretable framework for property prediction in materials informatics.
• To represent materials using human-readable text descriptions for machine learning models.
• To enhance the accessibility of advanced AI tools for materials scientists.

Main Methods:

• Utilized transformer language models pre-trained on a large corpus of scientific literature (2 million articles).
• Input materials data included chemical composition, crystal symmetry, and site geometry, translated into text-based representations.
• Employed local interpretability techniques to explain model predictions and ensure consistency with domain expertise.

Main Results:

• The proposed text-based representation and language model approach outperformed crystal graph networks in classifying four out of five material properties.
• Achieved high accuracy even with very limited datasets (ultra-small data limit) after fine-tuning.
• Generated explanations that are faithful to the model's internal workings and align with expert rationales.

Conclusions:

• A language-centric framework offers a transparent and accurate method for material property prediction.
• This approach democratizes access to advanced AI in materials informatics, benefiting users without AI expertise.
• Text-based representations combined with transformer models provide a powerful and interpretable alternative to traditional black-box models.