缩小差距:预期与部署绩效对比
Alice X Zhou1,2, Melissa D Aczon1,2, Eugene Laksana1,2
1Department of Anesthesiology and Critical Care Medicine, Children's Hospital Los Angeles, Los Angeles, California, USA.
概括
纵向数据分区最好估计临床环境中的反复神经网络模型性能. 在培训中包含较旧的数据并没有降低性能,确保了对患者护理的可靠预测模型.
科学领域:
- 临床信息学 临床信息学
- 医疗保健中的机器学习
- 预测建模预测建模
背景情况:
- 准确预测未来的表现对于成功的临床模型开发至关重要.
- 过于乐观的绩效估计可能导致在现实环境中不使用预测模型.
- 循环神经网络 (RNN) 模型越来越多地用于临床预测.
研究的目的:
- 评估不同的数据分区方法如何影响RNN模型的内部测试性能估计.
- 当内部测试性能与现实世界部署进行比较时,量化乐观 (高估性能).
- 评估将旧数据纳入培训套件对模型性能的影响.
主要方法:
- 利用儿科重症监护室 (2010-2020年) 的数据进行两个预测任务:ICU死亡率和双级阳性气道压力衰竭.
- 采用各种数据分区策略,包括纵向分区 (对较新的数据进行测试),以创建开发和测试集.
- 在历史数据 (2010-2018) 上训练可部署的RNN模型,并在随后的数据 (2019-2020) 上评估它们,以模拟现实世界的部署.
主要成果:
- 纵向分区方法表现出最不乐观的情况,提供了对未来性能更准确的估计.
- 在培训数据集中包含较旧的数据并没有对可部署模型的性能产生负面影响.
- 利用所有可用的数据来开发模型,最大限度地利用纵向分区的好处来进行年度绩效评估.
结论:
- 纵向数据分区是一种可靠的方法来估计临床预测模型的实际性能.
- 将较旧的数据纳入训练集是可行的,并且不会影响模型性能.
- 这些发现支持开发用于临床应用的强大和可靠的预测模型.
相关概念视频
01:01Production Efficiency
16.9K
Net production efficiency (NPE) is the efficiency at which organisms assimilate energy into biomass for the next trophic level. Due to low metabolic rates and less energy spent on thermoregulatory processes, the NPE of ectotherms (cold-blooded animals) is 10 times higher than endotherms (warm-blooded animals).
16.9K
01:19Testing a Claim about Standard Deviation
2.5K
A complete procedure to test a claim about population standard deviation or population variance is explained here.
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
2.5K
01:52Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
01:17Common Leveling Mistakes and Errors
103
A survey team is tasked with determining the elevation difference between points Point A and Point B, separated by uneven terrain. They use a leveling instrument and a leveling rod.Common MistakesMisreading the Rod: During a backsight reading at Point A, the instrumentman observes the rod partially obscured by tall grass. Instead of reading 1.135 m, they mistakenly record 1.735 m due to the misalignment of the crosshair with the wrong graduation. This error adds 0.600 m to all subsequent...
103
01:25Interpreting Run Charts
236
Run charts, essentially line graphs plotted over time, serve as fundamental yet effective tools for process analysis. They chronicle data sequentially, facilitating the identification of trends, shifts, or cyclical movements. This graphical representation is instrumental in determining whether a process is stable or exhibits signs of potential instability indicative of special cause variation. In the healthcare domain, run charts depict infection rates over time, enabling hospitals to monitor...
236
02:45Self-Discrepancy Theory
18.4K
One influential perspective on what motivates people's behavior is detailed in Tory Higgin's self-discrepancy theory (Higgins, 1987). He proposed that people hold disagreeing internal representations of themselves that lead to different emotional states.
18.4K