CLOUD: A Scalable and Physics-Informed Foundation Model for Crystal Representation Learning

Area of Science:

• Materials Science
• Computational Materials Science
• Crystallography

Background:

• Predicting crystal properties is crucial for materials discovery but is limited by resource-intensive experimental and computational methods.
• Existing machine learning models struggle with generalizability and interpretability due to data requirements and inadequate representation of structural characteristics.
• Integrating physics into machine learning models is challenging but essential for improved performance.

Purpose of the Study:

• To introduce CLOUD (Crystal Language mOdel for Unified and Differentiable materials modeling), a novel transformer-based framework for crystalline materials.
• To develop a scalable, physics-informed foundation model for accelerating materials discovery and property prediction.
• To demonstrate the potential of differentiable materials modeling for predicting temperature-dependent properties.

Main Methods:

• Developed CLOUD, a transformer framework utilizing Symmetry-Consistent Ordered Parameter Encoding (SCOPE) for a compact, coordinate-free representation of crystal symmetry, Wyckoff positions, and composition.
• Pre-trained CLOUD on over six million crystals and fine-tuned it on diverse downstream tasks for property prediction.
• Integrated CLOUD with the Debye model for differentiable prediction of phonon-related properties, ensuring thermodynamic consistency.

Main Results:

• CLOUD achieves competitive performance across various material properties, demonstrating scalability with increasing data and model size.
• The SCOPE representation effectively encodes essential structural characteristics, enhancing model generalizability.
• Physics-informed differentiable modeling enabled accurate, temperature-dependent phonon property prediction without additional data.

Conclusions:

• CLOUD offers a scalable and physics-informed foundation model for crystalline materials, unifying symmetry-consistent representations with physics-grounded learning.
• This approach accelerates materials discovery by enabling efficient and accurate prediction of material properties.
• The framework demonstrates the power of differentiable materials modeling for robust and interpretable predictions.