在美国退伍军人卫生管理局全国部署的人口健康风险算法中的性能偏差
Likhitha Kolla1, Kristin Linn1,2, Amol S Navathe1,2,3
1Perelman School of Medicine, University of Pennsylvania, Philadelphia.
JAMA health forum
|August 15, 2025
概括
退伍军人卫生管理局的护理评估需要 (CAN) 算法.
科学领域:
- 医疗信息学 医疗信息学
- 临床决策支持系统临床决策支持系统
- 在医疗保健中的预测建模.
背景情况:
- 临床风险算法对于决策支持和质量指标至关重要.
- 随着时间的推移,算法性能可能会降低,导致决策和资源分配中的错误.
- 退伍军人卫生管理局 (VA) 护理评估需求 (CAN) 算法,每年在全国范围内用于超过500万退伍军人,尚未评估其性能漂移.
研究的目的:
- 评估性能漂移对VA CAN算法的影响.
- 评估这些性能变化的程度,机制和临床后果.
- 了解算法漂移如何影响患者护理和资源分配.
主要方法:
- 使用VA电子健康记录和行政数据 (2016-2021) 的回顾性队列研究.
- 分析了来自700多万独一无二的退伍军人超过2700万的观察结果.
- 评估模型性能指标 (例如,TPR,FPR,PPV,NPV,F1得分,准确性) 的变化以及国家质量指标.
主要成果:
- 在2016年至2021年期间,VA CAN算法的性能下降,积极预测值 (PPV) 和F1得分显著下降,虚假阳性率 (FPR) 增加.
- 在包括人口统计和医疗保健利用在内的19个共同变量中观察到显著的变化.
- 国家质量指标显示,符合条件的退伍军人中FPR增加,尽管住院和死亡率下降.
结论:
- VA CAN算法的性能下降是由于结果流行率和共变量分布的变化造成的.
- 持续监测临床风险算法及其衍生的质量指标是必不可少的.
- 积极的监测可以帮助缓解低于最佳的资源分配,并改善临床决策.
更多相关视频
07:31Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
7.2K
06:05The Participant-Reported Implementation Update and Score PRIUS: A Novel Method for Capturing Implementation-Related Data Over Time
Published on: February 19, 2021
1.4K
相关概念视频
01:29Bias in Epidemiological Studies
666
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
666
03:33Genetic Drift
40.6K
Natural selection—probably the most well-known evolutionary mechanism—increases the prevalence of traits that enhance survival and reproduction. However, evolution does not merely propagate favorable traits, nor does it always benefit populations.
40.6K
01:12Relative Risk
334
Relative risk (RR) is a statistical measure commonly used in epidemiology to compare the likelihood of a particular event occurring between two groups. This metric is important for evaluating the relationship between exposure to a specific risk factor and the probability of a particular outcome. It plays a crucial role in medical research, public health studies, and risk assessment. Relative risk quantifies how much more (or less) likely an event is to occur in an exposed group compared to an...
334
01:12Analysis of Population Pharmacokinetic Data
381
Analysis of population pharmacokinetic data involves studying the behavior of drugs within diverse populations to understand their pharmacokinetic parameters. Traditional pharmacokinetic methods typically involve collecting samples from a few individuals and estimating these parameters. While these methods are commonly used, they have limitations in capturing the variability in drug response among individuals or heterogeneous populations. Population pharmacokinetics is employed to address these...
381
01:09Mutation, Gene Flow, and Genetic Drift
59.4K
In a population that is not at Hardy-Weinberg equilibrium, the frequency of alleles changes over time. Therefore, any deviations from the five conditions of Hardy-Weinberg equilibrium can alter the genetic variation of a given population. Conditions that change the genetic variability of a population include mutations, natural selection, non-random mating, gene flow, and genetic drift (small population size).
59.4K
01:12Hazard Ratio
240
The hazard ratio (HR) is a widely used measure in clinical trials to compare the risk of events, such as death or disease recurrence, between two groups over time. It reflects the ratio of hazard rates—the instantaneous risk of the event occurring—between a treatment group and a control group. This measure provides valuable insights into the relative effectiveness of a treatment by assessing how the risk of an event differs between the two groups.
For example, in a clinical trial...
For example, in a clinical trial...
240