标准和新型不确定性校准技术的推导和实验性能
Katherine E Brown1, Steve Talbert2, Douglas A Talbert3
1Vanderbilt University Medical Center, Nashville, TN.
概括
本研究引入了新的指标,即拒绝分类指数 (RC-Index) 和相对的RC-Index,用于评估机器学习模型中的不确定性量化. 这些指标提供了一种超越视觉分析的定量方法,用于评估AI应用中的模型可靠性.
科学领域:
- 机器学习 机器学习
- 人工智能在医学中的应用
- 不确定性定量化 不确定性定量化
背景情况:
- 机器学习模型的透明度至关重要,特别是在医疗AI中.
- 不确定性量化 (UQ) 测量模型无知,但通常是视觉评估.
- 现有的UQ评估方法存在局限性.
研究的目的:
- 开发用于评估UQ绩效的定量指标.
- 引入拒绝分类指数 (RC-指数) 和相对的RC-指数 (rRC-指数).
- 评估UQ指标识别不正确的ML预测的能力.
主要方法:
- 在现有的RC-Index上进行了扩展.
- 引入了新的相对RC指数 (rRC-Index).
- 使用排斥分类曲线作为这些不确定性指标的基础.
- 将RC-Index和rRC-Index与已建立的基于升空曲线的措施进行比较.
主要成果:
- RC-Index和rRC-Index提供了对UQ业绩的定量衡量.
- 这些指标来自拒绝分类曲线.
- 该研究建立了一个用于比较UQ评估指标的框架.
结论:
- 拒绝分类曲线为UQ评估提供了坚实的基础.
- RC-Index和rRC-Index为视觉分析提供了一个定量替代方案.
- 这些指标可以改善机器学习模型在关键应用中的可靠性评估.
相关概念视频
Instrument Calibration
687
Instrument calibration is essential for ensuring that instruments produce accurate and consistent results. It is vital in manufacturing, healthcare, testing laboratories, and scientific research. Calibration processes are specific to each instrument and help enhance data accuracy. Each instrument has a unique calibration process tailored to its design and function to improve data accuracy.
Analytical Balance Calibration
An analytical balance measures mass and requires regular calibration to...
Analytical Balance Calibration
An analytical balance measures mass and requires regular calibration to...
687
Uncertainty: Overview
1.6K
In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
1.6K
Uncertainty in Measurement: Reading Instruments
50.3K
Counting is the type of measurement that is free from uncertainty, provided the number of objects being counted does not change during the process. Such measurements result in exact numbers. By counting the eggs in a carton, for instance, one can determine exactly how many eggs are there in the carton. Similarly, the numbers of defined quantities are also exact. For example, 1 foot is exactly 12 inches, 1 inch is exactly 2.54 centimeters, and 1 gram is exactly 0.001 kilograms. Quantities...
50.3K
Uncertainty in Measurement: Accuracy and Precision
99.9K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
99.9K
Propagation of Uncertainty from Random Error
1.8K
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
1.8K
Propagation of Uncertainty from Systematic Error
1.4K
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
1.4K


