Knowledge distillation of neural network potential for molecular crystals

Area of Science:

• Materials Science
• Computational Chemistry
• Machine Learning

Background:

• Organic molecular crystals offer diverse functionalities but exploring them computationally is challenging.
• Quantum chemical calculations are accurate but slow; neural network potentials (NNPs) offer speed but risk bias from inorganic training data.

Purpose of the Study:

• To improve the accuracy of pre-trained NNPs for organic molecular crystals.
• To investigate knowledge distillation strategies for transferring information from a teacher model.

Main Methods:

• Fine-tuning a pre-trained NNP using knowledge distillation from a teacher model.
• Comparing the impact of soft targets (teacher inference values) versus hard targets (ground truth) on NNP accuracy.
• Evaluating the distilled NNP's performance in predicting elastic properties of organic crystals.

Main Results:

• Knowledge distillation using only soft targets proved most effective for improving NNP accuracy.
• Increasing the proportion of hard target loss led to decreased knowledge transfer efficiency and overfitting.
• The distilled NNP demonstrated improved accuracy in predicting elastic properties compared to the original pre-trained model.

Conclusions:

• Knowledge distillation, particularly with soft targets, is a viable strategy to adapt NNPs for organic molecular crystals.
• This approach mitigates bias and enhances predictive accuracy for materials properties.
• The developed NNP shows promise for accelerating the discovery of novel organic crystalline materials.