哈特-流:在视频中生成小组活动场景图的层次注意力流机制
Naga Venkata Sai Raviteja Chappa1, Pha Nguyen1, Thi Hoang Ngan Le1
1Department of EECS, University of Arkansas, Fayetteville, AR 72701, USA.
Sensors (Basel, Switzerland)
|June 19, 2024
概括
预测性视频场景理解得到了新的集团活动场景图生成数据集和层次注意力流 (HAtt-Flow) 机制的推进. 这种流动注意力方法改善了视频中的实时关系预测.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 组活动场景图 (GASG) 生成对于理解动态视频内容至关重要.
- 视频场景图形生成 (VidSGG) 的传统方法仅限于回顾性分析,阻碍了预测能力.
研究的目的:
- 引入GASG的新数据集,并提供详细的注释.
- 提出一个创新的分层注意力流 (HAtt-Flow) 机制,以提高GASG的性能.
- 为了推进预测性视频场景理解.
主要方法:
- 通过将JRDB数据集扩展到外观,交互,位置,关系和情况属性,开发了一个新的GASG数据集.
- 介绍了HAtt-Flow机制,将流量网络理论应用于注意力机制.
- 传统的注意力"价值"和"关键"被转化为"源头"和"沉"在注意力流框架内.
主要成果:
- 在GASG任务中,HAtt-Flow模型表现出显著的有效性.
- 拟议的流动注意力机制在现有方法上显示出优越性.
- 广泛的实验验证了模型的性能和方法的新性.
结论:
- 哈特-流机制在预测视频场景理解方面取得了重大进展.
- 开发的数据集丰富了复杂活动的场景理解能力.
- 这项工作为视频数据中的实时关系预测提供了有价值的技术.
相关概念视频
01:20Parallel Processing
150
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
150
01:42Schemas
11.6K
A schema is a mental construct consisting of a cluster or collection of related concepts (Bartlett, 1932). There are many different types of schemata, and they all have one thing in common: schemata are a method of organizing information that allows the brain to work more efficiently. When a schema is activated, the brain makes immediate assumptions about the person or object being observed.
11.6K