通过骨架推进人类运动识别CLIP++:加权视频特征集成和增强对比样本歧视
Lin Yuan1, Zhen He1, Qiang Wang1
1Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China.
Sensors (Basel, Switzerland)
|February 24, 2024
概括
SkeletonCLIP++ 通过语义信息和加权框架方法来增强人类行动识别. 它可以更好地区分类似的行动,并通过BERT文本集成来完善性能.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 传统的人类行为识别依赖于基于标签的方法.
- 现有的模型可能无法完全捕捉微妙的人类运动或语义相关性.
- 需要先进的技术来改善行动差异化.
研究的目的:
- 介绍SkeletonCLIP++,一个扩展的人类行动识别框架.
- 增强模型利用简单标签之外的语义信息的能力.
- 改进视频中人类行为的细微表现和差异化.
主要方法:
- 实现了"加权集成" (WFI) 用于微妙的视频功能计算.
- 引入了"对比样本识别" (CSI) 以加强行动歧视.
- 集成了预先训练的"BERT文本编码器集成" (BTEI) 来完善性能.
主要成果:
- 在HMDB-51,UCF-101和NTU RGB+D 60数据集上表现出积极的性能改进.
- 在较小的数据集上表现出特别的有效性.
- 在密切相关的人类行为之间实现了更详细的区分.
结论:
- 骨CLIP++提供了一种精致的方法来识别人类行动.
- 该模型确保了视频数据分析中的语义完整性和详细差异化.
- 这项工作通过将语义理解纳入行动识别模型来推动该领域的发展.
相关概念视频
Relative Motion Analysis using Rotating Axes
460
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
460
Absolute Motion Analysis- General Plane Motion
219
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
219
Relative Motion Analysis using Rotating Axes-Problem Solving
402
Consider a crane whose telescopic boom rotates with an angular velocity of 0.04 rad/s and angular acceleration of 0.02 rad/s2. Along with the rotation, the boom also extends linearly with a uniform speed of 5 m/s. The extension of the boom is measured at point D, which is measured with respect to the fixed point C on the other end of the boom. For the given instant, the distance between points C and D is 60 meters.
Here, in order to determine the magnitude of velocity and acceleration for point...
Here, in order to determine the magnitude of velocity and acceleration for point...
402
Kinematic Equations - II
9.5K
The second kinematic equation expresses the final position of an object in terms of its initial position, the distance traveled with the initial constant velocity, and the distance traveled due to a change in velocity. Similar to the first kinematic equation, this equation is also only valid when the acceleration is constant throughout the motion of an object.
Suppose a car merges into freeway traffic on a 200 m long ramp. If its initial velocity is 10 m/s and it accelerates at 2 m/s2, then the...
Suppose a car merges into freeway traffic on a 200 m long ramp. If its initial velocity is 10 m/s and it accelerates at 2 m/s2, then the...
9.5K
Relative Motion Analysis using Rotating Axes - Acceleration
336
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame. The absolute velocity of point B is determined by adding the absolute velocity of point A, the relative velocity of point B in the rotating frame, and the effects caused by the angular velocity within the rotating frame.
Time differentiation is...
Time differentiation is...
336
Kinematic Equations: Problem Solving
12.4K
When analyzing one-dimensional motion with constant acceleration, the problem-solving strategy involves identifying the known quantities and choosing the appropriate kinematic equations to solve for the unknowns. Either one or two kinematic equations are needed to solve for the unknowns, depending on the known and unknown quantities. Generally, the number of equations required is the same as the number of unknown quantities in the given example. Two-body pursuit problems always require two...
12.4K


