在临床预测模型中存在可疑研究实践的证据
Nicole White1, Rex Parsons1, Gary Collins2
1Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Kelvin Grove, Queensland, Australia.
BMC medicine
|September 4, 2023
概括
研究人员可能通过重新分析数据来实现所需的曲线下面面积 (AUC) 值来"破解"临床预测模型. 这种做法会膨胀模型的性能,可能会损害患者的护理.
科学领域:
- 医疗信息学 医疗信息学
- 生物统计学 生物统计学
- 医疗保健服务研究 医疗服务研究
背景情况:
- 临床预测模型在医疗保健研究中至关重要.
- 曲线下的面积 (AUC) 量化了模型的区分能力.
- 预定义的AUC值 (0.7,0.8,0.9) 可能会激励数据操纵.
研究的目的:
- 调查公布的AUC值中"破解"的证据.
- 分析潜在操纵模式的AUC值的分布.
主要方法:
- 从PubMed摘要中提取了AUC值.
- 使用具有0.01个容器的组图来可视化AUC分布.
- 将观察到的AUC分布与平滑的脊线分布进行比较.
主要成果:
- 观察到AUC值在0.7,0.8和0.9值以上显著超过.
- 在低于这些值的AUC值中确定了相应的不足.
- 分析包括306,888个AUC值.
结论:
- 研究结果表明,在临床预测模型中,AUC值可能会被"破解".
- 过度膨胀的AUC可能会导致患者的治疗水平低于最佳水平.
- 通过共享协议,数据和代码提高透明度至关重要.
相关概念视频
Bias in Epidemiological Studies
349
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
349
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
147
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
147
Confounding in Epidemiological Studies
188
Confounding in statistical epidemiology represents a pivotal challenge, referring to the distortion in the perceived relationship between an exposure and an outcome due to the presence of a third variable, known as a confounder. This variable is associated with both the exposure and the outcome but is not a direct link in their causal chain. Its presence can lead to erroneous interpretations of the exposure's effect, either exaggerating or underestimating the true association. This...
188
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
Sensitivity, Specificity, and Predicted Value
485
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
485
Hazard Ratio
155
The hazard ratio (HR) is a widely used measure in clinical trials to compare the risk of events, such as death or disease recurrence, between two groups over time. It reflects the ratio of hazard rates—the instantaneous risk of the event occurring—between a treatment group and a control group. This measure provides valuable insights into the relative effectiveness of a treatment by assessing how the risk of an event differs between the two groups.
For example, in a clinical trial...
For example, in a clinical trial...
155


