MSP:多模式自我注意力促成学习
概括
多模式即时学习有效地适应视觉语言模型. 新的多模式自我注意提示 (MSP) 框架通过直接修改注意力机制,克服先前输入级适应方法的局限性,改善了对新课程的概括性.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 多模式即时学习使像CLIP这样的视觉语言模型 (VLM) 适应新任务.
- 传统方法在输入层面调整VLM,导致几何扭曲并限制概括.
- 这种间接的适应导致了新课程的过度装配和性能降低,称为间接学习困境 (ILD).
研究的目的:
- 引入一种新的框架,即多式联机自动注意提示 (MSP),以实现高效和可通用的VLM适应.
- 通过将适应从输入转移到语义层来克服间接学习困境 (ILD).
- 提高VLM在下游任务中的性能和稳定性,特别是对于新型类别.
主要方法:
- 开发了多模式自我注意提示 (MSP) 框架.
- 注入可学习提示直接进入变压器注意力块的键和值序列.
- 集成的远程意识优化和部分提示学习 (随机维度掩盖) 提高了适应性和稳定性.
主要成果:
- 在11个基准中实现了80.67%的最先进的和平均精度.
- 在新课程上表现出卓越的性能,准确率为77.32%,比以前的方法绝对提高2.18%.
- 仅需要0.11万个可学习参数,在11个数据集中的10个数据集中超过了CLIP的零射击性能.
结论:
- MSP框架有效地解决了多式联络快速学习中的间接学习困境.
- 直接调节的注意力机制可以实现精确的下游适应,同时保留预训练的嵌入.
- MSP为VLMs的高效,强大和可通用的基于提示的适应建立了新的范式.
更多相关视频
08:45A Dual Task Procedure Combined with Rapid Serial Visual Presentation to Test Attentional Blink for Nontargets
Published on: December 5, 2014
9.6K
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
8.0K
相关概念视频
Purposive Learning
447
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
447
Observational Learning
841
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
841
Associative Learning
1.3K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.3K
Cognitive Learning
1.0K
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
1.0K
Self-Presentation: Self-Monitoring and Self-Handicapping
43.5K
People can go to great lengths to protect their self-image and present themselves in ways that they want others to see them. Sociologist Erving Goffman presented the idea that a person is like an actor on a stage. Calling his theory dramaturgy, Goffman believed that we use “impression management” to present ourselves to others as we hope to be perceived. Each situation is a new scene, and individuals perform different roles depending on who is present (Goffman, 1959). Think about...
43.5K
Chunking and Rehearsal in Sensory Memory
562
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
562
