在视频中识别人与物体的互动,并对自然语言进行监督
Qiyue Li1, Xuemei Xie2, Jin Zhang2
1School of Communications and Information Engineering, Xi'an University of Posts and Telecommunications, Xi'an, Shaanxi 710121, PR China.
概括
本研究引入了一种新的自然语言监督学习模型,用于在视频中识别人与物体交互 (HOI). 通过利用类别文本中的语义特征,该模型显著提高了HOI识别能力.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 自然语言处理自然语言处理.
背景情况:
- 传统的人物对象交互 (HOI) 识别模型主要使用视觉数据,并将任务视为多重分类问题,忽视语义标签信息.
- 现有的方法忽视了HOI类别在自然语言描述中存在的丰富的语义关系.
研究的目的:
- 为HOI识别提出一种自然语言监督学习模型,该模型包含来自类别文本的语义信息.
- 通过使用自然语言来监督视觉特征学习来增强视觉特征表示.
主要方法:
- 引入了人与物体交互类别的文本特征,作为自然语言监督学习模型的标签.
- 应用了对比式学习范式,使用图像-文本配对预训练模型来提取单个图像和文本特征.
- 使用时空混合模块生成高语义,基于组合的HOI特征.
- 将视觉交互特征与类别文本特征进行比较,以基于相似性的推断.
主要成果:
- 在两个HOI数据集上实现了最先进的性能.
- 在CAD-120数据集上显示出高的F1分数:93.6%的子活动和93.1%的负担能力.
结论:
- 拟议的模型有效地利用HOI类别标签文本中的语义信息.
- 从多模式预训练模型中利用大规模的图像-文本配对数据,提高视觉-文本对应性和HOI识别精度.
相关概念视频
Automatic Processing and Automatic Social Behavior
2
Automatic processing refers to the cognitive operations that occur without conscious intent or awareness, playing a fundamental role in shaping social cognition and behavior. These processes enable individuals to navigate complex social environments efficiently by relying on mental shortcuts and pre-existing knowledge structures known as schemas. One of the most influential mechanisms underlying automatic processing is priming, which subtly activates mental representations through exposure to...
2
Observational Learning
321
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
321
Natural and Artificial Concepts
270
In psychology, concepts can be divided into two categories: natural and artificial. Natural concepts are formed through direct or indirect experiences. For example, consider the concept of snow. If you live in a place with regular snowfall, such as Essex Junction, Vermont, you know snow through direct experiences. You’ve seen it fall, touched it, shoveled it, and played in it. You recognize its texture, appearance, and even its smell. In contrast, if you live on an island like Saint...
270


