多重语义提示符,用于没有排练的持续学习
Junwei Chen1, Zhenyu Zhang1, Depeng Li1
1School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, 430074, China.
概括
多重语义提示 (MSP) 通过结合提示来保持跨任务的知识,有效地解决深度学习中的灾难性遗忘问题. 这种方法提高了零射击传输性能,而不需要先前的数据,显著提高了准确性.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算机视觉 计算机视觉
背景情况:
- 深度神经网络面临灾难性的遗忘,在学习新任务时失去旧任务的性能.
- 持续学习试图减轻这一问题,即时基于的方法作为一个有希望的解决方案而出现,而无需重复数据.
- 现有的提示方法可能无法充分利用语义信息来实现强大的持续学习.
研究的目的:
- 开发一种新的基于提示的持续学习方法,可以增强语义信息的保留.
- 在持续学习环境中,提高视觉语言模型的零射击传输能力.
- 克服传统数据重复和基本快速方法的局限性.
主要方法:
- 拟议的多重语义提示 (MSP) 方法将提示和文本标签连接起来以捕获语义信息.
- 结合提示基于视觉和文本特征之间的相似性.
- 使用预先训练的模型进行零射击转移,由组合提示指令指导.
主要成果:
- MSP表现出优于基于提示的最先进方法的性能,平均最终准确度高出3%以上.
- 与基准上的香草视觉语言模型相比,实现了超过15%的更高的零射击分类准确性.
- 在任务之间有效地保留知识,而不需要排练缓冲区.
结论:
- 多重语义提示 (MSP) 方法是一种高度有效的持续学习方法,可以显著缓解灾难性遗忘.
- MSP增强了语义理解,并提高了零射击传输能力,为数据重播提供了保护隐私的替代方案.
- 提出的方法代表了基于提示的持续学习策略的重大进步.
更多相关视频
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
7.7K
09:22A Method for Remotely Silencing Neural Activity in Rodents During Discrete Phases of Learning
Published on: June 22, 2015
14.8K
相关概念视频
Elaborative Rehearsals
134
Elaborative rehearsal is a crucial cognitive strategy that strengthens information encoding in long-term memory by making meaningful connections between new data and pre-existing knowledge. This approach contrasts with maintenance rehearsal, which involves simple repetition without delving into the significance of the information. While maintenance rehearsal might temporarily keep information active in short-term memory, it is less effective for long-term retention.
The effectiveness of...
The effectiveness of...
134
Chunking and Rehearsal in Sensory Memory
298
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
298
Cognitive Learning
531
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
531
Purposive Learning
207
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
207
Observational Learning
317
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
317
Associative Learning
586
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
586
