Uncertainty Calibration in Molecular Machine Learning: Comparing Evidential and Ensemble Approaches

Area of Science:

• Quantum Chemistry
• Machine Learning
• Computational Chemistry

Background:

• Machine learning (ML) models are vital in quantum chemistry.
• Reliability of ML models depends on uncertainty quantification (UQ).
• Existing UQ methods like deep evidential regression (DER) and deep ensembles have limitations.

Purpose of the Study:

• Compare DER and deep ensembles for UQ in quantum chemistry.
• Evaluate the impact of post hoc calibration on UQ methods.
• Assess the effectiveness of calibration techniques on QM9 and WS22 datasets.

Main Methods:

• Applied deep evidential regression (DER) and deep ensembles.
• Utilized post hoc calibration methods: isotonic regression (ISR), standard scaling, and GP-Normal.
• Evaluated performance on QM9 and WS22 datasets for molecular property prediction.

Main Results:

• Raw uncertainties from DER and ensembles were miscalibrated.
• Calibration techniques successfully aligned predicted variances with observed errors.
• Calibrated DER improved high-confidence prediction filtering on QM9.
• Calibrated ensembles reduced redundant ab initio evaluations by over 20% on WS22, enhancing active learning.

Conclusions:

• Post hoc calibration is essential for accurate UQ in quantum chemistry ML.
• Calibration transforms uncertainty estimates into actionable insights.
• Calibrated UQ methods ensure trustworthy predictions and resource-efficient molecular modeling.