Crystal Composition Transformer: Self-Learning Neural Language Model for Generative and Tinkering Design of...

Area of Science:

• Artificial intelligence
• Materials science
• Computational chemistry

Background:

• Self-supervised neural language models excel in biological and molecular sequence analysis.
• Existing models often lack interpretability and are not optimized for generative design.
• Masking-based pre-trained models struggle with the unique challenges of materials design.

Purpose of the Study:

• Introduce a novel probabilistic generative model for inorganic materials design.
• Develop a model capable of both generating new materials and suggesting modifications (tinkering).
• Enhance interpretability and data efficiency in AI-driven materials discovery.

Main Methods:

• Developed a Blank-filling Language Model for Materials (BLMM) Crystal Transformer.
• Adapted blank-filling techniques from natural language processing to learn materials grammars.
• Utilized unsupervised transformer language models for AI-based generative design.

Main Results:

• Achieved high chemical validity in generated materials (89.7% charge neutrality, 84.8% balanced electronegativity).
• Demonstrated superior performance over pseudo-random sampling baselines.
• Successfully applied BLMM to discover new materials validated by Density Functional Theory (DFT) calculations.

Conclusions:

• BLMM offers a powerful, interpretable, and data-efficient approach to generative materials design.
• The model facilitates materials tinkering and doping through learned chemical principles.
• This work integrates unsupervised transformer language models into inorganic materials discovery, with a supporting web application available.