融合注意力对动作识别:集成稀疏密度和全球注意力对视频动作识别
1Department of Artificial Intelligence Application, Hanyang University, Seoul 04763, Republic of Korea.
Sensors (Basel, Switzerland)
|November 9, 2024
概括
融合注意力对动作识别 (FAR) 通过结合稀疏密度和全局注意力模式来增强视频分析. 这种方法提高了时间特征捕获和动作识别效率,优于传统方法.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 传统的视频动作识别通常使用全球关注,这与视频中的时间冗余性作斗争.
- 蒙面视频建模预培训方法有希望,但可以过度强调空间而不是时间特征.
研究的目的:
- 通过有效利用时间特征,提出Fusion Attention for Action Recognition (FAR) 的新方法,通过有效利用时间特征来改善动作识别.
- 为了减轻处理视频时间冗余的全球关注的局限性.
主要方法:
- 在微调过程中,FAR将稀疏密度的注意力模式 (专注于时间特征) 与全局注意力融合在一起.
- 关键组件包括头部分裂稀疏密集注意力 (HSDA),令牌组交互,以及组平均分类器.
- HSDA将注意力分为整合全球和稀疏密度模式,专注于管状补丁进行时间分析.
主要成果:
- 与 Something-Something V2 和 Kinetics-400 数据集的全球注意力微调相比,FAR 实现了更高的性能 (+0.1-0.4%),使用更少的计算.
- 定性分析证明了FAR在高冗余视频中捕捉时间特征的有效性.
- 该方法利用来自VideoMAE和MVD的预训练重量.
结论:
- FAR提供了一种高效和有效的视频动作识别方法,特别是在有时间冗余的场景中表现出色.
- 拟议的方法改善了时间信息的利用,从而提高了识别精度.
- 未来的研究可以探索FAR的适应性与各种预培训方法.
相关概念视频
01:21Association Areas of the Cortex
5.1K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
5.1K
01:22Multi-input and Multi-variable systems
98
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
98