用视觉语言模型进行基于频率的全面快速学习
IEEE transactions on pattern analysis and machine intelligence
|August 19, 2025
概括
本研究介绍了基于频率的全面提示学习 (FCPrompt),通过生成视觉概念的详细文本提示来增强视觉语言模型 (VLM). FCPrompt 提高了 VLM 对下游任务的可转移性.
科学领域:
- 计算机视觉 计算机视觉
- 自然语言处理自然语言处理.
- 机器学习 机器学习
背景情况:
- 基于变压器的视觉语言模型 (VLM) 显示出强大的性能,但缺乏具有区分性的本地视觉特征.
- 现有的方法很难从变压器视觉编码器中提取具有代表性的本地信息.
- 预先训练有素的VLM需要在各种下游应用中提高可转移性.
研究的目的:
- 开发一种学习全面文本提示的方法,这些提示描述从粗到细的视觉概念.
- 增强预先训练有素的VLM的可转移性,用于各种下游任务.
- 解决从基于变压器的VLM中提取本地视觉信息的挑战.
主要方法:
- 建议基于频率的全面快速学习 (FCPrompt) 来提取本地视觉信息.
- 使用离散等边变换 (DCT) 将视觉编码器特征转换为频率域.
- 采用双分支框架,为全球和基于频率的本地视觉信息调整提供优化的文本提示.
主要成果:
- FCPrompt有效地从多余的特征中挖掘出具有代表性的本地视觉信息.
- 该方法使用DCT获得了紧的,信息丰富的和解的局部视觉表示.
- 学习文本提示全面描述从粗到细的视觉概念.
结论:
- FCPrompt显著提高了VLMs的描述能力.
- 该方法在多个基准指标中实现了最先进的性能.
- 该方法提供了一种新的方法来提高VLM的适应性和下游任务的性能.
更多相关视频
10:11Portable Intermodal Preferential Looking IPL: Investigating Language Comprehension in Typically Developing Toddlers and Young Children with Autism
Published on: December 14, 2012
18.6K
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
681
相关概念视频
Introduction to Learning
530
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
530
Learning Disabilities
271
Learning disabilities are cognitive disorders caused by neurological impairments that affect cognitive functions like language and reading, without indicating overall intellectual or developmental challenges. These disabilities differ from global intellectual or developmental disabilities as they are limited to distinct cognitive functions. Common learning disabilities include dysgraphia, dyslexia, and dyscalculia, each of which impacts unique aspects of learning.
Dyslexia
Dyslexia is a...
Dyslexia
Dyslexia is a...
271
Language Development
447
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
447
Purposive Learning
206
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
206
Observational Learning
311
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
311
Vision
55.3K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
55.3K
