Uncertainty Prediction for Machine Learning Models of Material Properties.

Area of Science:

• Materials Science
• Artificial Intelligence
• Computational Materials Science

Background:

• Uncertainty quantification (UQ) is vital for reliable AI in materials science.
• Confidence intervals are common for ML models, but individual prediction intervals are less frequent.
• Accurate prediction intervals enhance trust and applicability of AI-driven material discovery.

Purpose of the Study:

• To compare three distinct methods for calculating individual prediction intervals in AI-based material property predictions.
• To evaluate the advantages and disadvantages of each UQ approach.
• To identify the most effective method for reliable uncertainty estimation in materials informatics.

Main Methods:

• Comparison of three approaches: quantile loss function, direct prediction interval modeling, and Gaussian processes.
• Testing on 12 machine learning (ML)-physical properties using data from the JARVIS-DFT database.
• Development and availability of codes within the JARVIS-tools package for prediction interval computation.

Main Results:

• Each method demonstrated unique strengths and weaknesses for UQ.
• Direct modeling of prediction intervals proved easiest to implement.
• Direct modeling generally minimized over- and underestimation of predicted errors across tested properties.

Conclusions:

• Directly modeling individual prediction intervals is a promising approach for AI in materials science.
• This method offers a balance of simplicity and accuracy in quantifying prediction uncertainty.
• The developed tools facilitate the integration of robust UQ into materials informatics workflows.