通过双面知识蒸在轻量级模型中增强少数镜头学习
Bojun Zhou1, Tianyu Cheng2, Jiahao Zhao1
1School of Information Science and Technology, Nantong University, Nantong 226019, China.
Sensors (Basel, Switzerland)
|March 28, 2024
概括
这项研究引入了一种新的知识蒸方法,将知识从大型计算机视觉模型转移到轻量级模型,改善资源有限的环境中的少数镜头分类性能.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
背景情况:
- 大规模模型主导着计算机视觉分类,但由于资源限制,面临部署挑战.
- 轻量级模型提供了效率,但在性能方面往往落后,特别是在少数拍摄的学习场景中.
研究的目的:
- 为资源有限的环境增强轻量级模型的分类性能.
- 开发一个有效的知识蒸策略,以从大模型转移到小模型的知识.
主要方法:
- 提出了一种双面的知识蒸策略,将基于输出和基于中间特征的蒸结合起来.
- 集成的特征错误分布校准,以解决中间特征中的非高斯特征偏差.
- 在MiniImageNet,CIFAR-FS和CUB数据集上评估了该方法,用于少数镜头分类任务.
主要成果:
- 拟议的方法显著提高了轻量级模型在少数拍摄分类中的性能.
- 与基准数据集上现有的最先进的轻量级模型相比,表现出优异的结果.
- 在挑战五向一拍和五向五拍学习任务方面取得了显著的收益.
结论:
- 双面知识蒸策略有效地将知识压缩到轻量级模型中.
- 这种方法可以在资源有限的设备上进行高效和高性能的几次分类.
- 该方法为在实际应用中部署先进的计算机视觉功能提供了一个有希望的解决方案.
相关概念视频
Observational Learning
170
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
170
Generalization, Discrimination, and Extinction
548
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
548
Associative Learning
353
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
353
Introduction to Learning
379
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
379
Purposive Learning
119
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
119
Cognitive Learning
239
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
239


