通过层级相关性传播优化,通过层级相关性传播优化,提高深度神经网络的概括性和背景偏差的稳定性
Pedro R A S Bassi1,2, Sergio S J Dertkigil3, Andrea Cavalli4,5
1Alma Mater Studiorum - University of Bologna, Bologna, Italy. pedro.salvadorbassi2@unibo.it.
Nature communications
|January 4, 2024
概括
本研究介绍了ISNet,这是一种减少深度学习模型背景偏差的方法,防止捷径学习. ISNet优化了层层的相关性传播热图,增强了对现实应用的模型概括性.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 图像数据集中的背景偏差可以导致虚假的相关性,导致深度神经网络 (DNN) 中的快捷方式学习.
- 这种现象导致模型在训练数据上表现良好,但对现实世界的场景概括不佳.
- 例如医疗成像,胸部X射线中的背景特征可以偏向COVID-19或结核病等疾病的分类.
研究的目的:
- 开发一种方法,尽量减少背景偏差,并阻碍深度分类器中的快捷方式学习.
- 改善DNN在外部数据集上的概括性能.
- 为背景偏差提供一个计算高效和广泛适用的解决方案.
主要方法:
- 该研究提出了一种方法 (ISNet),可以优化层级相关性传播 (LRP) 热图以减轻背景偏差.
- 合成背景偏差被注入到图像数据集中进行定量比较.
- ISNet的性能与八个先进的DNN在偏向和外部数据集上进行了评估.
主要成果:
- 与八个基准DNN相比,ISNet表现出了对背景偏差的优越稳定性.
- 该方法通过关注相关的图像特征,如胸部X射线中的肺部,显著减少了快捷方式的学习.
- 在非分布式测试数据库上,ISNet实现了显著更好的泛化性能.
结论:
- 优化LRP热图是一种有效的策略,可以在深度学习模型中对抗背景偏差和快捷方式学习.
- 在不增加计算成本的情况下,ISNet提供了一种轻便,快速和广泛适用的解决方案.
- 拟议的方法提高了DNN的可靠性和现实应用性,特别是在医学诊断等关键领域.
相关概念视频
01:24Generalization, Discrimination, and Extinction
565
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
565
02:07Improving Translational Accuracy
10.6K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
10.6K
01:18Reducing Line Loss
154
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
154
01:10Propagation of Uncertainty from Systematic Error
522
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
522
00:59Propagation of Uncertainty from Random Error
691
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
691
01:05Difference from Background: Limit of Detection
6.4K
The limit of detection (LOD) is the smallest amount of analyte that can be distinguished from the background noise. The LOD value corresponds to the concentration at which the analyte signal is three times larger than the standard deviation of the blank signal. Below this value, the analyte signal cannot be differentiated from the background noise. It is calculated by dividing the calibration slope by 3 times the standard deviation of the blank signals.
The LOD indicates the presence or absence...
The LOD indicates the presence or absence...
6.4K