知识蒸与标签噪音相遇 学习:模糊性引导的相互标签炼油厂
IEEE transactions on neural networks and learning systems
|November 29, 2023
概括
本研究引入了以模糊性为指导的相互标签炼油厂知识蒸 (AML-KD) 来训练使用噪音数据的较小模型. AML-KD有效地改进了标签,使学生模型即使有不完美的数据集也能够准确.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算机视觉 计算机视觉
背景情况:
- 知识蒸 (KD) 将知识从大型教师模型转移到较小的学生模型.
- 现有的KD方法经常与含有噪音标签的真实数据集扎,导致性能下降.
- 处理噪音标签对于强大而实用的深度学习应用程序至关重要.
研究的目的:
- 为了应对知识蒸与杂的标签的挑战.
- 提出一种新的方法,即以模糊性为指导的相互标签炼油厂知识蒸 (AML-KD),用于在标签噪音下培训学生模型.
- 开发一个强大的框架,提高学生模型的准确性,尽管标签不完美.
主要方法:
- 提出了一个两阶段的标签精炼框架,利用预先培训的教师模型.
- 第一阶段涉及标签传播 (LP),由教师模型指导的小损失选择.
- 第二阶段使用教师和学生模型之间的相互LP,结合一个模两可的重量估计 (AWE) 模块来处理模两可的样本.
主要成果:
- AML-KD在培养准确和高效的学生模型方面表现出有效性,即使标签噪声很大.
- 合成和现实世界噪声数据集的实验结果显示,与最先进的KD和标签噪声学习 (LNL) 方法相比,性能优越.
- 该AWE模块成功地缓解了标签精制过程中模糊样本造成的过拟问题.
结论:
- 在有噪音的标签的情况下,AML-KD为知识蒸提供了一个强大的解决方案.
- 拟议的方法可以从不完美的数据创建高精度,低成本的学生模型.
- AML-KD通过为现实世界杂数据集提供实用方法,推进知识蒸领域.
相关概念视频
Labeling Emotion
143
Emotional labeling is a cognitive process that involves identifying and naming one's emotions, such as anger, fear, happiness, or sadness. It allows individuals to recognize and express their internal emotional states, a critical aspect of emotional regulation and communication. Labeling emotions requires more than mere recognition; it also involves drawing upon memory and contextual cues to understand the current situation and apply a corresponding emotional label. For instance, feeling...
143
Purposive Learning
121
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
121
Labeling DNA Probes
8.2K
DNA probes are fragments of DNA labeled with a reporter tag to enable their detection or purification. The resulting labeled DNA probes can then hybridize to target nucleic acid sequences through complementary base-pairing, and may be used to recover or identify these regions.
Radioisotopes, fluorophores, or small molecule binding partners like biotin or digoxigenin, are the most widely used reporter tags for labeling DNA probes. These labels can be attached to the probe DNA molecule via...
Radioisotopes, fluorophores, or small molecule binding partners like biotin or digoxigenin, are the most widely used reporter tags for labeling DNA probes. These labels can be attached to the probe DNA molecule via...
8.2K
Difference from Background: Limit of Detection
6.4K
The limit of detection (LOD) is the smallest amount of analyte that can be distinguished from the background noise. The LOD value corresponds to the concentration at which the analyte signal is three times larger than the standard deviation of the blank signal. Below this value, the analyte signal cannot be differentiated from the background noise. It is calculated by dividing the calibration slope by 3 times the standard deviation of the blank signals.
The LOD indicates the presence or absence...
The LOD indicates the presence or absence...
6.4K
Cognitive Learning
246
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
246
[3,3] Sigmatropic Rearrangement of Allyl Vinyl Ethers: Claisen Rearrangement
2.1K
The Claisen rearrangement is a [3,3] sigmatropic rearrangement of allyl vinyl ethers to unsaturated carbonyl compounds. The rearrangement is a concerted pericyclic reaction proceeding via a chair-like transition state.
2.1K


