视频swin-CLSTM变压器:通过光流和长期依赖来增强人类行动识别
Jun Qin1,2, Shenwei Chen1,3, Zheng Ye1
1College of Computer Science, South-Central Minzu University, Wuhan, Hubei, China.
PloS one
|July 7, 2025
概括
本研究介绍了视频Swin-CLSTM变压器,用于改进人类动作识别 (HAR). 该模型通过整合光流和ConvLSTM来提高准确性和效率,在UCF-101数据集上表现优于现有的方法.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 视频数据量正在迅速增加,使人动识别 (HAR) 对于监控和体育分析等应用至关重要.
- 当前的深度学习 HAR 方法与微妙的动作,复杂的背景,长期依赖和计算效率作斗争.
研究的目的:
- 开发一种先进的HAR模型,克服现有的深度学习方法的局限性.
- 提高视频中识别人类行为的准确性和效率,特别是在具有挑战性的条件下.
主要方法:
- 视频Swin-CLSTM变压器模型是使用视频Swin变压器骨干开发的.
- 采用光流信息和稀疏采样策略来处理背景干扰.
- 嵌入了卷积长期短期记忆 (ConvLSTM) 单元以捕捉长期的行动依赖.
主要成果:
- 该模型在UCF-101数据集上实现了92.8%的Top-1和99.4%的Top-5精度,分别超过了基线3.2%和2.0%.
- 与没有光流的模型相比,计算成本平均降低了3.3%.
- 除研究证实了光流 (2.6%的精度增加) 和ConvLSTM (1.9%的精度增加) 的显著贡献.
结论:
- 与其他基于Swin变压器的方法相比,视频Swin-CLSTM变压器在HAR任务中表现出卓越的性能.
- 集成光流和ConvLSTM模块有效地提高了动作识别的准确性和效率.
- 在ImageNet-1K-LSTM上进行定制预训练进一步提高了性能,这表明它对HAR的价值.
相关概念视频
01:24Absolute Motion Analysis- General Plane Motion
273
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
273
01:25Relative Motion Analysis using Rotating Axes
543
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
543