在交叉验证之前应用过量采样将导致放射学学中的高偏差
1Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Hufelandstraße 55, 45147, Essen, Germany. aydin.demircioglu@uk-essen.de.
Scientific reports
|May 21, 2024
概括
在交叉验证之前对放射性数据进行过量抽样的不正确应用会导致模型性能和校准的显著偏差. 再抽样方法只应应用于训练数据,以确保准确的结果.
科学领域:
- 医疗成像医学成像
- 机器学习 机器学习
- 数据科学数据科学数据科学
背景情况:
- 类不平衡在临床放射性数据集中很常见,可能会影响分类器培训.
- 用重新抽样技术 (过量抽样,不足抽样) 来解决类不平衡问题.
- 再抽样的不正确应用可能导致数据泄露和错误的性能评估.
研究的目的:
- 通过对放射性数据错误地应用重新采样方法来量化引入的偏差.
- 用正确与不正确的重新采样策略训练的预测模型的性能进行比较.
- 评估数据不平衡对偏差大小的影响.
主要方法:
- 在15个放射性数据集上使用了5倍的交叉验证30次重复.
- 我们比较了两个培训场景:在交叉验证过程中正确的重新抽样与在交叉验证之前不正确的重新抽样.
- 评估偏差使用接收器操作特征曲线 (AUC) 下的面积,灵敏度,特异性,平衡精度和Brier分数.
- 进行了模拟研究以验证.
主要成果:
- 错误的过量抽样导致了实质性的积极偏差 (高达0.34的AUC,0.33的灵敏度,0.31的特异性,0.37的平衡精度).
- 偏差随着数据不平衡而增加,每次不平衡增加约有0.10的AUC增加.
- 校准偏差 (布赖尔分数) 达到了 -0.18.
- 低采样方法显示没有显著的偏差.
结论:
- 在交叉验证之前,对整个数据集应用过量采样是不正确的,在放射性模型评估中引入了显著的偏差.
- 在交叉验证中,重新采样方法必须仅适用于训练数据,以防止数据泄露,并确保可靠的性能指标.
- 低样本似乎不太容易受到这种特定类型的偏差的影响.
相关概念视频
Bias in Epidemiological Studies
246
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
246
Improving Translational Accuracy
10.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
10.1K
Receiver Operating Characteristic Plot
138
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
138
Sensitivity, Specificity, and Predicted Value
289
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
289
Downsampling
154
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
154
Ultrasonography
4.5K
Ultrasonography is an imaging technique that uses high-frequency sound waves to visualize the body's internal structures. It is a non-invasive and safe procedure that does not involve the use of ionizing radiation, making it widely used in various medical fields. Ultrasonography is used to study heart function, blood flow in the neck or extremities, certain conditions such as gallbladder disease, and fetal growth and development.
During an ultrasonography procedure, a handheld device called...
During an ultrasonography procedure, a handheld device called...
4.5K


