可通用视觉语言模型的结构诱导梯度调节
IEEE transactions on pattern analysis and machine intelligence
|September 1, 2025
概括
渐变调节的Meta-prompt学习 (GRAM) 增强了视觉语言模型的快速调整,通过Meta-learning和渐变调节提高了几次拍摄的适应性和跨领域的概括性.
科学领域:
- 人工智能
- 计算机视觉
- 自然语言处理
背景情况:
- 快速调整使用软提示有效地调整预先训练的视觉语言模型.
- 在初始化敏感性和过度调整方面面临挑战.
- 现有的方法在缺乏数据的情况下难以快速适应和保持通用性.
研究的目的:
- 引入一个新的渐变调节的元提示学习 (GRAM) 框架.
- 在少量和零射击学习场景中提高提示调整的有效性.
- 提高跨领域的概括性,减少视觉语言模型的过度匹配.
主要方法:
- 使用弱标记的图像文本数据开发了一种元学习范式.
- 用于数据组织的交叉模式层次聚类.
- 引入了一个渐变调节函数,以提高通用性.
- 为测试时间调整提出了一个结构诱导的梯度调节功能.
主要成果:
- GRAM实现了最先进的几次射击和零次射击通用性.
- 该框架不断改进各种快速调整方法.
- 有限或未标记的数据证明了有效的适应.
- 在不同领域展示强大的超级学习.
结论:
- 对于快速调整的挑战,GRAM提供了强大而适应性的解决方案.
- 提出的方法显著提升了几次射击和零次射击学习能力.
- 格拉姆提供了一种不依赖于模型的方法来提高即时调性能.
- 在没有明确注释的情况下,该框架促进了高效的知识转移.
相关概念视频
Language and Cognition
438
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
438
Higher Mental Functions of the Brain: Language
1.0K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
1.0K
Language Development
444
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
444
Visual System
684
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
684
Vision
55.3K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
55.3K
Neural Regulation
39.9K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
39.9K


