一个模型评估问题:通过认识不确定性量化确定难以识别的子组.
Katherine E Brown1,2, Steve Talbert3, Douglas A Talbert1
1Tennessee Technological University, Cookeville, TN.
AMIA ... Annual Symposium proceedings. AMIA Symposium
|January 15, 2024
概括
使用规则解释机器学习的不确定性可以识别模型表现良好或不佳的患者子组. 这增强了对不同数据段的模型行为的信任和理解.
科学领域:
- 机器学习 机器学习
- 人工智能的人工智能
- 医疗数据分析 医学数据分析
背景情况:
- 机器学习 (ML) 中的不确定性量化 (UQ) 提供了对模型可靠性的洞察,并建立了信任.
- 精心校准的UQ将高不确定性与增加的分类错误联系在一起.
- 目前的UQ主要关注个人预测确定性.
研究的目的:
- 调查解释ML模型不确定性的规则是否可以识别具有不同性能的子组.
- 为了确定UQ是否可以在患者亚群中提供对模型行为的全球理解.
- 为了评估UQ的实用性,超越个人预测信心.
主要方法:
- 开发基于规则的ML模型不确定性的解释.
- 将技术应用于深度神经网络和渐变增强合奏.
- 在基准和现实世界医疗数据集上评估方法.
主要成果:
- 生成的规则成功地划分了分类不确定性高和低的子组.
- 这些规则对应于模型表现出不同的性能水平的子组.
- 该方法在不同的ML架构和数据集中表现出有效性.
结论:
- 通过规则解释ML不确定性可以在全球范围内描述患者子组的模型性能.
- UQ的实用性从个体预测评估延伸到对子群体的模型行为理解.
- 这种方法提高了医学应用中的ML模型的可信度和可解释性.
相关概念视频
Mechanistic Models: Compartment Models in Individual and Population Analysis
43
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
43
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
56
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
56
Quantifying and Rejecting Outliers: The Grubbs Test
1.6K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.6K
Uncertainty: Confidence Intervals
4.1K
The confidence interval is the range of values around the mean that contains the true mean. It is expressed as a probability percentage. The interpretation of a 95% confidence interval, for instance, is that the statistician is 95% confident that the true mean falls within the interval. The upper and lower limits of this range are known as confidence limits. The confidence limits for the true mean are estimated from the sample's mean, the standard deviation, and the statistical factor...
4.1K
Strategies for Assessing and Addressing Confounding
101
Confounding is a critical issue in epidemiological studies, often leading to misleading conclusions about associations between exposures and outcomes. It occurs when the relationship between the exposure and the outcome is mixed with the effects of other factors that influence the outcome. Given that, addressing confounding is of high importance for drawing accurate inferences in research.
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
101
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation
513
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
On...
513


