全球和本地语义完成学习为视觉语言预培训
概括
本研究介绍了视觉语言预训练 (VLP) 模型的全球和本地语义完成学习 (GLSCL). 通过同时学习全球和本地特征,GLSCL增强了跨模式对齐,改善了各种基准的表现.
科学领域:
- 人工智能的人工智能
- 计算机视觉 计算机视觉
- 自然语言处理自然语言处理.
背景情况:
- 对视觉语言预训练 (VLP) 模型来说,交叉模式的调整至关重要.
- 在VLP中现有的蒙面建模任务主要集中在局部-局部对齐上,忽视了全球语义特征.
- 这种限制阻碍了全球表示与其他模式的本地特征的调整.
研究的目的:
- 为VLP提出一个新的全球和本地语义完成学习 (GLSCL) 任务.
- 同时促进全球-本地和本地-本地对齐.
- 增强VLP模型的交叉模式对齐能力.
主要方法:
- 引入了GLSCL任务,包括掩盖全球语义完成 (MGSC) 和掩盖本地令牌完成 (MLTC).
- MGSC专注于学习具有代表性的全球特征,而MLTC重建模态融合本地代币.
- 开发了一种灵活的视觉编码器,用于同时执行图像-文本和视频-文本任务,并引入了ALIGN-BENCH验证基准.
主要成果:
- 拟议的GLSCL任务有效地补充了缺失的语义,并通过跨模式交互恢复全球和本地特征.
- MGSC增强了全球特征表示,影响下游任务性能.
- 通过重建本地代币,MLTC提高了对多式联运数据的理解.
结论:
- 通过解决全球和本地特征学习,GLSCL显著改善了跨模式对齐.
- 该方法在各种视觉语言基准上实现了最先进的性能,包括视觉问题答案和图像文本检索.
- 灵活的视觉编码器和ALIGN-BENCH有助于评估和推进VLP模型.
相关概念视频
01:22Language Development
450
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
450
02:07Improving Translational Accuracy
2.7K
2.7K
01:12Observational Learning
314
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
314
01:27Language and Cognition
442
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
442
01:22Purposive Learning
207
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
207
01:18Introduction to Learning
532
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
532