在存在标签选择的情况下,避免偏见的临床机器学习模型性能估计
Conor K Corbin1,2, Michael Baiocchi3, Jonathan H Chen2,4,5
1Department of Biomedical Data Science, Stanford, California, USA.
概括
估计临床机器学习模型的性能需要仔细考虑部署人口. 这项研究表明,标签选择偏差性能指标,但因果推理权重估计器可以恢复全人口的准确估计.
科学领域:
- 机器学习 机器学习
- 临床信息学 临床信息学
- 因果推理因果推理
背景情况:
- 评估临床机器学习模型需要了解部署人口.
- 标签选择,观察到的患者是部署人口的子集,可能导致误导性绩效估计.
- 由于有偏见的标签选择,标准指标可能不会准确地反映现实世界的表现.
研究的目的:
- 描述标签选择类别和模拟场景,以评估机器学习性能指标中的偏差.
- 调查标签选择机制如何影响模型歧视和校准.
- 提出和评估用于在部署的临床模型中获得准确的性能估计的方法.
主要方法:
- 模拟了标签选择的五种因果上不同的场景.
- 在通常报告的二进制机器学习性能指标中评估偏差.
- 应用因果推理的传统权重估计器.
- 训练有素的机器学习模型来标记低产量的实验室诊断.
- 提出了一个更改的部署程序,将随机化和加权估计结合起来.
主要成果:
- 受观察特征影响的选择可能会误导歧视估计.
- 受标签影响的选择可能会误导校准估计.
- 权重估计器,当正确指定的,恢复完整的人口估计.
- 对于观察到的人口,AUROC的天真估计在实际任务中低于实际表现的高达20%.
- 拟议的部署程序恢复了真正的模型性能.
结论:
- 标签选择对准确的临床机器学习模型评估构成重大挑战.
- 因果推断方法,特别是权重估计器,对于纠正绩效偏差至关重要.
- 误导性绩效估计可能导致有益的临床工具的过早终止.
- 结合随机化和权重的更改部署策略有效地恢复了真正的模型性能.
相关概念视频
Pharmacokinetic Models: Comparison and Selection Criterion
109
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
109
Sensitivity, Specificity, and Predicted Value
507
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
507
Prediction Intervals
2.3K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.3K
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
148
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
148
Bias in Epidemiological Studies
375
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
375
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
84
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
84


