对比临床预测模型的超参数调整程序:一个模拟研究.
Zoë S Dunias1, Ben Van Calster2,3, Dirk Timmerman2,4
1Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands.
Statistics in medicine
|January 8, 2024
概括
标准交叉验证 (5倍或10倍) 有效地调整了临床预测模型,优于其他方法. 避免使用一个标准错误规则 (1SE CV) 进行交叉验证,因为它可能会损害模型校准.
科学领域:
- 生物统计学 生物统计学
- 医疗保健中的机器学习
- 临床预测建模临床预测建模
背景情况:
- 超参数调整对于优化临床风险预测模型至关重要.
- 通常使用的方法是Ridge,Lasso,弹性网和随机森林.
- 评估调整程序对于可靠的样本外性能至关重要.
研究的目的:
- 系统地比较临床预测模型的超参数调整程序.
- 评估样本大小,预测数和事件分数对调性能的影响.
- 评估各种调方法的区分,校准和预测错误.
主要方法:
- 在低维数据上进行了广泛的模拟.
- 比较,拉索,弹性网和随机森林模型.
- 评估了标准交叉验证 (5 倍,10 倍,重复),引导和一个标准错误规则 (1SE CV).
主要成果:
- 标准交叉验证 (5倍和10倍) 证明了优越的校准和整体性能.
- 1SE CV规则经常导致严重的校准错误.
- 引导调整显示出比标准CV更严重的误校准倾向.
- 在较小的样本大小和较低的事件分数下,表现的差异更为明显.
结论:
- 建议使用标准的5倍或10倍交叉验证来调整临床预测模型以尽量减少预测错误.
- 由于潜在的校准问题,在低维设置中应谨慎使用1SE CV规则.
- 调整程序的选择显著影响模型的预测性能,特别是在具有挑战性的数据场景中.
相关概念视频
Comparing the Survival Analysis of Two or More Groups
195
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
195
Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data
130
Statistical inference techniques, paramount in hypothesis testing, differentiate into two broad categories: parametric and nonparametric statistics.
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
130
Hazard Ratio
128
The hazard ratio (HR) is a widely used measure in clinical trials to compare the risk of events, such as death or disease recurrence, between two groups over time. It reflects the ratio of hazard rates—the instantaneous risk of the event occurring—between a treatment group and a control group. This measure provides valuable insights into the relative effectiveness of a treatment by assessing how the risk of an event differs between the two groups.
For example, in a clinical trial...
For example, in a clinical trial...
128
Receiver Operating Characteristic Plot
183
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
183
Statistical Methods to Analyze Parametric Data: Student t-Test and Goodness-of-Fit Test
1.6K
In parametric statistics, two fundamental tests stand out for their utility and wide application: the Student's t-test and goodness-of-fit tests. These tests provide researchers with a robust method for drawing insights from data, testing hypotheses, and making informed decisions based on their findings.
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with...
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with...
1.6K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
56
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
56


