Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

226
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
226
Estimation of the Physical Quantities01:05

Estimation of the Physical Quantities

7.1K
On many occasions, physicists, other scientists, and engineers need to make estimates of a particular quantity. These are sometimes referred to as guesstimates, order-of-magnitude approximations, back-of-the-envelope calculations, or Fermi calculations. The physicist Enrico Fermi was famous for his ability to estimate various kinds of data with surprising precision. Estimating does not mean guessing a number or a formula at random. Instead, estimation means using prior experience and sound...
7.1K
Uncertainty: Overview00:59

Uncertainty: Overview

1.4K
In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
1.4K
Propagation of Uncertainty from Systematic Error01:10

Propagation of Uncertainty from Systematic Error

1.2K
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
1.2K
Propagation of Uncertainty from Random Error00:59

Propagation of Uncertainty from Random Error

1.6K
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
1.6K
Mechanistic Models: Compartment Models in Individual and Population Analysis01:23

Mechanistic Models: Compartment Models in Individual and Population Analysis

195
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
195

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Erratum: Probabilistic performance estimators for computational chemistry methods: Systematic improvement probability and ranking probability matrix. I. Theory [J. Chem. Phys. 152, 164108 (2020)].

The Journal of chemical physics·2020
Same author

Models and corrections: Range separation for electronic interaction-Lessons from density functional theory.

The Journal of chemical physics·2020
Same author

Probabilistic performance estimators for computational chemistry methods: Systematic improvement probability and ranking probability matrix. II. Applications.

The Journal of chemical physics·2020
Same author

Erratum: "Probabilistic performance estimators for computational chemistry methods: The empirical cumulative distribution function of absolute errors" [J. Chem. Phys. 148, 241707 (2018)].

The Journal of chemical physics·2019

Related Experiment Video

Updated: Dec 22, 2025

Computation of Atmospheric Concentrations of Molecular Clusters from ab initio Thermochemistry
12:11

Computation of Atmospheric Concentrations of Molecular Clusters from ab initio Thermochemistry

Published on: April 8, 2020

8.6K

Probabilistic performance estimators for computational chemistry methods: Systematic improvement probability and

Pascal Pernot1, Andreas Savin2

  • 1Institut de Chimie Physique, UMR8000, CNRS, Université Paris-Saclay, 91405 Orsay, France.

The Journal of Chemical Physics
|May 3, 2020
PubMed
Summary

Evaluating computational chemistry theories requires new methods beyond standard error rankings. This study introduces a systematic improvement probability score and robust statistical indicators to assess ranking reliability, addressing benchmark dataset limitations.

More Related Videos

Isotopic Effect in Double Proton Transfer Process of Porphycene Investigated by Enhanced QM/MM Method
05:51

Isotopic Effect in Double Proton Transfer Process of Porphycene Investigated by Enhanced QM/MM Method

Published on: July 19, 2019

6.6K
A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

2.8K

Related Experiment Videos

Last Updated: Dec 22, 2025

Computation of Atmospheric Concentrations of Molecular Clusters from ab initio Thermochemistry
12:11

Computation of Atmospheric Concentrations of Molecular Clusters from ab initio Thermochemistry

Published on: April 8, 2020

8.6K
Isotopic Effect in Double Proton Transfer Process of Porphycene Investigated by Enhanced QM/MM Method
05:51

Isotopic Effect in Double Proton Transfer Process of Porphycene Investigated by Enhanced QM/MM Method

Published on: July 19, 2019

6.6K
A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

2.8K

Area of Science:

  • Computational chemistry
  • Theoretical chemistry
  • Chemical physics

Background:

  • Standard mean unsigned error rankings in computational chemistry are limited by non-normal error distributions and trends.
  • Existing complementary statistics like error quantiles and prediction uncertainty partially address these limitations.
  • Uncertainty arising from incomplete benchmark datasets is often overlooked in method evaluation.

Purpose of the Study:

  • To introduce a novel scoring system, the systematic improvement probability, for direct system-wise comparison of absolute errors.
  • To develop robust statistical indicators, inversion probability (P_inv) and ranking probability matrix (P_r), to quantify ranking uncertainty.
  • To highlight the importance of correlations between error sets in statistical comparisons.

Main Methods:

  • Direct system-wise comparison of absolute errors to calculate systematic improvement probability.
  • Development of robust statistical indicators: inversion probability (P_inv) and ranking probability matrix (P_r).
  • Analysis of correlations between error sets to assess their impact on ranking statistics.

Main Results:

  • The proposed systematic improvement probability offers a new metric for evaluating computational chemistry methods.
  • P_inv and P_r provide essential measures of ranking robustness against benchmark dataset incompleteness.
  • Correlations between error sets significantly influence the reliability of statistical comparisons.

Conclusions:

  • The standard mean unsigned error is insufficient for robustly ranking computational chemistry methods.
  • Novel statistical approaches, including systematic improvement probability and robust indicators (P_inv, P_r), enhance the evaluation of theoretical models.
  • Accurate assessment of computational chemistry theories necessitates considering error distributions, dataset incompleteness, and error correlations.