Related Experiment Video
Updated: Aug 30, 2025
10:22
Split Point Analysis and Uncertainty Quantification of Thermal-Optical Organic/Elemental Carbon Measurements
Published on: September 7, 2019
8.3K
A universal similarity based approach for predictive uncertainty quantification in materials science
Vadim Korolev1, Iurii Nevolin2, Pavel Protsenko3
1Department of Chemistry, Lomonosov Moscow State University, Moscow, 119991, Russia. korolewadim@gmail.com.
Scientific Reports
|September 2, 2022
Summary
A new uncertainty quantification (UQ) method, the Δ-metric, is introduced for machine learning (ML) models in materials informatics. This universal approach accurately ranks predictive errors, offering a cost-effective solution for diverse ML applications.
Area of Science:
- Materials Informatics
- Machine Learning
- Computational Science
Background:
- Significant advancements in machine learning (ML) accuracy have been made, yet robust uncertainty quantification (UQ) remains a challenge.
- Current UQ methods are often model-specific or rely on complex ensemble techniques, necessitating a universal approach.
- There is a critical need for a UQ method applicable to single ML models across various algorithms.
Purpose of the Study:
- To introduce a novel, universal uncertainty quantification (UQ) measure, the Δ-metric, for machine learning models.
- To address the limitations of existing model-specific or ensemble-based UQ methods.
- To provide a readily applicable UQ technique for diverse ML algorithms in materials informatics.
Main Methods:
- Developed the Δ-metric, a quantitative criterion inspired by the k-nearest neighbor approach used in chemoinformatics for applicability domain estimation.
- Applied the Δ-metric to assess uncertainty in machine learning models across various material classes, ML algorithms, and input feature types.
- Compared the performance of the Δ-metric against established UQ methods.
Main Results:
- The Δ-metric demonstrated superior performance in accurately ranking predictive errors compared to several existing UQ methods.
- The proposed UQ measure proved effective across diverse materials, machine learning algorithms, and input feature types, highlighting its universality.
- The Δ-metric offers a low-cost alternative and potential complement to advanced deep ensemble strategies for UQ.
Conclusions:
- The Δ-metric represents a significant advancement in uncertainty quantification for machine learning in materials informatics.
- Its universality and effectiveness across various scenarios make it a valuable tool for assessing the reliability of ML predictions.
- The Δ-metric provides a practical and efficient solution for enhancing the trustworthiness of machine learning models in scientific research.
Related Concept Videos
01:10Propagation of Uncertainty from Systematic Error
777
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
777
00:59Uncertainty: Overview
842
In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
842
00:59Propagation of Uncertainty from Random Error
971
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
971
04:08The Uncertainty Principle
23.7K
Werner Heisenberg considered the limits of how accurately one can measure properties of an electron or other microscopic particles. He determined that there is a fundamental limit to how accurately one can measure both a particle’s position and its momentum simultaneously. The more accurate the measurement of the momentum of a particle is known, the less accurate the position at that time is known and vice versa. This is what is now called the Heisenberg uncertainty principle. He...
23.7K
03:37Uncertainty in Measurement: Accuracy and Precision
76.4K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
76.4K
01:05Estimation of the Physical Quantities
5.2K
On many occasions, physicists, other scientists, and engineers need to make estimates of a particular quantity. These are sometimes referred to as guesstimates, order-of-magnitude approximations, back-of-the-envelope calculations, or Fermi calculations. The physicist Enrico Fermi was famous for his ability to estimate various kinds of data with surprising precision. Estimating does not mean guessing a number or a formula at random. Instead, estimation means using prior experience and sound...
5.2K