How to Train a Shallow Ensemble

Area of Science:

• Computational Materials Science
• Machine Learning
• Quantum Chemistry

Background:

• Shallow ensembles offer efficient uncertainty quantification for machine learning interatomic potentials due to shared model weights.
• Balancing calibration performance and computational cost in training shallow ensembles is crucial for practical applications.

Purpose of the Study:

• To systematically investigate training strategies for shallow ensembles to optimize uncertainty quantification.
• To evaluate efficient protocols for training shallow ensembles, minimizing computational overhead while maintaining calibration quality.

Main Methods:

• Explicit optimization of negative log-likelihood (NLL) loss for improved calibration.
• Modeling force uncertainties via NLL objective for reliable calibration.
• Full-model fine-tuning of pre-trained shallow ensembles as an efficient training protocol.

Main Results:

• Explicit NLL optimization enhances calibration compared to random initialization or Laplace approximation.
• Training solely on energy objectives leads to miscalibrated force estimates.
• Full-model fine-tuning achieves calibration quality comparable to training from scratch, reducing training time by up to 96%.

Conclusions:

• Explicitly modeling force uncertainties via NLL is essential for reliable calibration in machine learning potentials.
• Full-model fine-tuning presents an efficient and effective protocol for training shallow ensembles.
• Practical guidelines are established for reliable uncertainty quantification across diverse materials in atomistic machine learning.