更少的信心,更少的忘记:在无模范的课堂中与更谦逊的老师学习 增量学习
Zijian Gao1, Kele Xu1, Huiping Zhuang2
1National University of Defense Technology, Changsha 410000, China; State Key Laboratory of Complex & Critical Software Environment, Changsha 410000, China.
概括
阶级增量学习 (CIL) 面临灾难性遗忘 (CF). 我们提出与谦虚的教师学习 (LwHT) 解决过度自信的教师在无模范的CIL,提高模型的可塑性和性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算机视觉 计算机视觉
背景情况:
- 阶级增量学习 (CIL) 对于终身学习至关重要,但受到灾难性遗忘 (CF) 的影响.
- 知识蒸 (KD) 是一种常见的方法,通过使用以前的模型作为教师来缓解CIL中的CF.
- 没有实例的CIL场景,缺乏对旧数据的访问,加剧了CF,并给KD带来了独特的挑战.
研究的目的:
- 调查教师模型在无样本CIL中的过度自信现象.
- 在没有访问旧培训样本的情况下分析知识蒸 (KD) 的影响.
- 通过使用更合适的教师模型,提出一种新的方法来缓解灾难性遗忘 (CF).
主要方法:
- 进行实证实验和理论分析,以了解教师过度信任无实例的CIL.
- 开发了一种新的方法,Humbler教师学习 (LwHT),它选择一个合适的检查点模型作为一个不太自信的教师.
- 使用核规范规范化用于时间组合,以增强模型稳定性.
主要成果:
- 识别并分析了教师模型在无实例CIL中的过度自信行为.
- 证明LwHT在不同设置中显著优于最先进的方法10.41%,6.56%和4.31%.
- 与现有方法相比,展示了优越的模型可塑性和稳定性.
结论:
- 拟议的学习与谦虚的教师 (LwHT) 有效地解决灾难性遗忘 (CF) 在无示例的班级增量学习 (CIL).
- 选择"谦虚"的教师模式,并采用时间组合,可以提高学习稳定性和绩效.
- LwHT为强大的终身学习系统提供了一个有希望的方向.
更多相关视频
06:35Examining Recall Memory in Infancy and Early Childhood Using the Elicited Imitation Paradigm
Published on: April 28, 2016
33.9K
07:59Using Practice Testing, Public Speaking, and Source Monitoring to Examine the Influences of Learning Strategies and Stress on Episodic Memory
Published on: June 14, 2019
7.9K
相关概念视频
Observational Learning
158
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
158
Purposive Learning
108
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
108
Introduction to Learning
359
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
359
Avoidance Learning and Learned Helplessness
1.7K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
1.7K
Cognitive Learning
237
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
237
Associative Learning
332
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
332
