不确定性的不确定性? 对化学数据集的不确定性量化指标进行比较
Maria H Rasmussen1, Chenru Duan2,3, Heather J Kulik2,3
1Department of Chemistry, University of Copenhagen, Copenhagen, Denmark. mhr@chem.ku.dk.
Journal of cheminformatics
|December 19, 2023
概括
在化学研究中评估机器学习 (ML) 模型不确定性需要强大的指标. 这项研究比较了流行的验证指标,建议基于错误的校准可靠的不确定性量化.
科学领域:
- 计算化学是一种计算化学.
- 机器学习应用程序 机器学习应用程序
- 科学验证科学验证
背景情况:
- 机器学习 (ML) 模型在化学研究中至关重要,需要可靠的不确定性量化来进行预测.
- 现有的不确定性估计方法缺乏标准化的评估指标,阻碍了跨研究的一致评估.
研究的目的:
- 为了比较流行验证指标对化学ML模型不确定性估计的有效性.
- 引入用于解释负日志概率 (NLL) 和斯皮尔曼等级相关系数等指标的参考值.
- 评估莱维等人提出的基于错误的校准方法. (传感器2022年) 的项目.
主要方法:
- 斯皮尔曼等级相关系数,负日志概率 (NLL) 和错误校准区域与基于错误的校准进行比较.
- 引入模拟的错误参考值,以将度量解释置于背景.
- 使用玩具模型对测试套件设计的度量灵敏度的分析.
主要成果:
- 负日志概率 (NLL) 和斯皮尔曼等级相关系数在没有参考值的情况下提供了有限的洞察力.
- 基于错误的校准图提供了一个更全面的验证不确定性估计.
- 基于排名的指标,如斯皮尔曼的,对测试集组成非常敏感,导致结果变化.
结论:
- 基于错误的校准通常是验证化学研究中ML模型不确定性的最有效方法.
- 在使用基于排名的验证指标时,仔细考虑测试组设计是必不可少的.
- 对于可靠的化学ML应用,需要对不确定性量化方法进行标准化评估.
更多相关视频
12:11Computation of Atmospheric Concentrations of Molecular Clusters from ab initio Thermochemistry
Published on: April 8, 2020
8.2K
09:04Identifying Per- and Polyfluorinated Chemical Species with a Combined Targeted and Non-Targeted-Screening High-Resolution Mass Spectrometry Workflow
Published on: April 18, 2019
12.5K
相关概念视频
Uncertainty: Overview
560
In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
560
Propagation of Uncertainty from Systematic Error
528
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
528
Data Validation
164
Method validation is a crucial process in analytical chemistry designed to confirm that a given method consistently produces reliable and high-quality results. This process is essential when a method is applied to different sample matrices or when procedural modifications are made, ensuring that the results meet acceptable standards across various applications.
Key parameters for method validation include:
Key parameters for method validation include:
164
Uncertainty in Measurement: Accuracy and Precision
73.7K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
73.7K
Uncertainty in Measurement: Reading Instruments
38.2K
Counting is the type of measurement that is free from uncertainty, provided the number of objects being counted does not change during the process. Such measurements result in exact numbers. By counting the eggs in a carton, for instance, one can determine exactly how many eggs are there in the carton. Similarly, the numbers of defined quantities are also exact. For example, 1 foot is exactly 12 inches, 1 inch is exactly 2.54 centimeters, and 1 gram is exactly 0.001 kilograms. Quantities...
38.2K
Statistical Analysis: Overview
6.6K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
6.6K
