MS25: Materials Science-Focused Benchmark Data Set for Machine Learning Interatomic Potentials

Area of Science:

• Materials Science
• Computational Chemistry
• Machine Learning

Background:

• Machine learning interatomic potentials (MLIPs) are crucial for simulating materials behavior.
• Evaluating MLIPs requires diverse, materials-relevant benchmark datasets.
• Existing benchmarks may not fully capture the complexities of disordered or catalytic systems.

Purpose of the Study:

• To introduce MS25, a comprehensive benchmark dataset for MLIP evaluation.
• To compare the performance of five MLIP architectures across various material systems.
• To assess the reliability of MLIP predictions for physical observables beyond simple energy and force errors.

Main Methods:

• Developed the MS25 benchmark dataset encompassing MgO surfaces, liquid water, zeolites, catalytic Pt, high-entropy alloys (HEAs), and Zr-oxides.
• Trained and tested five MLIP architectures: MACE, NequIP, Allegro, MTP, and Torch-ANI.
• Evaluated MLIPs on traditional metrics (energies, forces, stresses) and derived physical observables (lattice constants, volumes, reaction barriers).

Main Results:

• Most MLIPs achieved comparable accuracy on simple systems, but equivariant models outperformed non-equivariant ones by 1.5-2x on complex/disordered systems (HEAs, Zr-O).
• Low energy/force errors did not guarantee accurate physical observables, underscoring the need for explicit validation.
• Significant limitations in cross-framework transferability were observed for zeolite models.
• HEAs and Zr-O datasets proved challenging, differentiating MLIP architectures effectively.

Conclusions:

• Benchmarking should prioritize understanding MLIP failure modes, transferability, and impact on observables over marginal accuracy gains.
• Equivariant MLIPs are recommended for complex material systems.
• For simpler problems, factors like computational cost, inference speed, and software integration are key decision drivers.