基于知识蒸的补充标签学习
Peng Ying1, Zhongnian Li1, Renke Sun1
1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou 221116, China.
Mathematical biosciences and engineering : MBE
|December 5, 2023
概括
本研究介绍了基于知识蒸 (KDCL) 的补充标签增强,这是改善弱监督学习的新框架. KDCL使用教师-学生模型增强了互补标签,提高了分类准确性.
科学领域:
- 机器学习 机器学习
- 人工智能的人工智能
- 计算机科学 计算机科学
背景情况:
- 补充标签学习 (CLL) 是一种弱监督的方法,使用非目标类信息.
- 现有的CLL方法往往在补充标签中的监管信号使用不足.
- 增强这些信号对于改善CLL性能至关重要.
研究的目的:
- 提出一个新的框架,基于知识蒸 (KDCL) 的补充标签增强,以丰富CLL的监督.
- 解决当前CLL方法在利用补充标签信息方面的局限性.
- 用补充标签训练的模型提高准确性.
主要方法:
- 介绍了KDCL,一个使用教师-学生深度神经网络架构的框架.
- 教师模式软化了补充标签,以加强监督.
- 学生模型从这些软化标签中学习,两者都在补充标签的数据上接受训练.
主要成果:
- 与基线CLL方法相比,KDCL优化的模型在四个数据集 (MNIST,F-MNIST,K-MNIST,CIFAR-10) 中显示出更高的准确性.
- 实验使用了各种数据集和教师-学生模型对 (Lenet-5+MLP,DenseNet-121+ResNet-18).
- 提出的方法有效地提高了分类性能.
结论:
- 通过有效加强监督,KDCL在补充标签学习方面取得了重大进展.
- 在KDCL内部的知识蒸方法成功地丰富了补充标签信息.
- 这一框架为改善低监督学习任务提供了一个有希望的方向.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
232
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
7.6K
相关概念视频
01:27Associative Learning
404
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
404
03:31Labeling DNA Probes
8.2K
DNA probes are fragments of DNA labeled with a reporter tag to enable their detection or purification. The resulting labeled DNA probes can then hybridize to target nucleic acid sequences through complementary base-pairing, and may be used to recover or identify these regions.
Radioisotopes, fluorophores, or small molecule binding partners like biotin or digoxigenin, are the most widely used reporter tags for labeling DNA probes. These labels can be attached to the probe DNA molecule via...
Radioisotopes, fluorophores, or small molecule binding partners like biotin or digoxigenin, are the most widely used reporter tags for labeling DNA probes. These labels can be attached to the probe DNA molecule via...
8.2K
01:24Generalization, Discrimination, and Extinction
572
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
572
01:22Purposive Learning
121
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
121
01:12Observational Learning
182
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
182
01:21Cognitive Learning
246
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
246