相关实验视频
Updated: May 29, 2025

05:37
An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
2.0K
临床预测模型的AUROC的不稳定性
Florian D van Leeuwen1, Ewout W Steyerberg1, David van Klaveren2,3
1Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands.
Statistics in medicine
|February 8, 2025
概括
临床预测模型 (CPM) 的外部验证显示了AUC等性能指标的显著变化. 经验贝叶斯方法更好地量化了这种不确定性,改善了模型验证和预期设置.
科学领域:
- 生物统计学 生物统计学
- 临床流行病学临床流行病学
- 医疗信息学 医疗信息学
背景情况:
- 外部验证对于在部署之前评估临床预测模型 (CPM) 性能至关重要.
- 患者群体差异和预测指标定义等因素影响模型的区分能力,通常用曲线下的面积 (AUC) 来量化.
- 在外部验证研究中量化AUC变化对于设定现实的性能预期至关重要.
研究的目的:
- 量化心血管疾病CPM的外部验证研究中AUC的变化.
- 提出在新环境中调整CPM绩效预期的方法.
- 为了比较一个经验贝叶斯的方法与频率的方法来估计AUC不确定性.
主要方法:
- 从Tufts-PACE CPM注册表中分析了469个CPM的AUC估计值,并进行了1603次外部验证.
- 使用随机效应元分析估计了AUC的研究间标准偏差 ().
- 开发了一种经验贝叶斯式方法,使用的逻辑正常分布作为先验,通过交叉验证进行比较.
主要成果:
- 每个CPM的外部验证的中位数为2 (IQR [1-3]).
- 估计分布的平均值为0.055 (SD 0.015).
- 频率主义方法低估了AUC不确定性,特别是在不到5次验证的情况下;贝叶斯方法实现了更好的覆盖.
结论:
- 验证的AUC的显著异质性导致预测未来绩效的不可减少的不确定性.
- 现有的方法往往低估了这种不确定性.
- 拟议的经验贝叶斯方法提供了一个强大的方法来判断预测模型的有效性,并保证了广泛的应用.
相关概念视频
Receiver Operating Characteristic Plot
72
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
72
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
Prediction Intervals
2.2K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.2K
Sensitivity, Specificity, and Predicted Value
168
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
168
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
114
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
114
Uncertainty in Measurement: Accuracy and Precision
73.2K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
73.2K

