层次的班扎夫互动通用视频语言表示学习学习的交互.
概括
本研究介绍了层次的Banzhaf交互 (HBI) 用于细粒度的视频语言表示学习. 使用游戏理论,HBI模拟视频-文本交互,改善对人工智能任务的多式联络理解.
科学领域:
- 人工智能的人工智能
- 计算机视觉 计算机视觉
- 自然语言处理自然语言处理.
背景情况:
- 多模式表示学习,特别是对比学习,在AI中至关重要.
- 视频语言表示学习目前依赖于粗的全球语义交互.
- 需要精细的多式联络学习来提高表现质量.
研究的目的:
- 为细粒度的视频语言表示学习引入一种新的方法.
- 用多变量合作游戏理论来模拟视频-文本交互,以处理不确定性.
- 开发一种方法,捕捉不同的细粒度,灵活的组合,和含义互动的模糊强度.
主要方法:
- 模拟视频文本作为游戏玩家使用多变量合作游戏理论.
- 从层次的角度设计层次的Banzhaf交互 (HBI) 来模拟视频剪辑和文本词之间的细粒度对应.
- 通过融合单模和跨模组件来重建表示,以减轻偏差并保留自适应编码.
- 将结构扩展为灵活的编码器-解码器框架,以适应下游任务.
主要成果:
- 层次Banzhaf交互有效模拟细粒度的语义对应.
- 代表重建减轻了偏差,确保了细粒度和自适应编码.
- 编码器-解码器框架在各种任务中表现出灵活性.
- 关于文本-视频检索,视频问答和视频标题基准的实验显示出卓越的性能.
结论:
- 拟议的方法显著提高了细粒度的视频语言表示学习.
- 这种方法可以有效地处理不确定性和各种语义相互作用.
- 该方法在多个基准上表现出强大的有效性和概括能力.
更多相关视频
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
475
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.6K
相关概念视频
Associative Learning
276
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
276
Higher Mental Functions of the Brain: Language
707
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
707
Hierarchy of Motor Control
2.4K
The hierarchy of motor control refers to the different levels of organization and processing involved in controlling movement in the body. These levels range from higher cortical areas involved in planning and decision-making to lower spinal cord reflexes that respond automatically to external stimuli.
2.4K
Language and Cognition
313
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
313
Multi-input and Multi-variable systems
93
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
93
Lateralization
299
Brain lateralization refers to the division of mental processes and functions between the two hemispheres of the brain, a phenomenon that optimizes neural efficiency and underpins complex abilities in humans. This specialization allows each hemisphere to perform tasks where it has a comparative advantage, facilitating more refined cognitive capabilities across different domains.
299
