文本控制的运动马巴:人体运动的文本指导的时间接地
概括
我们介绍了基于文本的人类运动接地 (THMG) 以在视频中使用文本来定位动作. 我们的TM-Mamba模型通过整合全局上下文和文本控制来有效处理长序列,在新的BABEL-Grounding数据集上取得了强有力的结果.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 对于机器人和动画等应用来说,理解人类运动至关重要.
- 现有的文字动作任务包括生成和编辑,但时间本地化仍然具有挑战性.
- 变压器模型由于二次计算成本而难以处理长序列.
研究的目的:
- 引入基于文本的人类运动接地 (THMG) 以实现精确的时间段定位.
- 解决变压器处理长,未修剪的动作序列的计算挑战.
- 提出一个新的,高效的模型,用于文本控制的运动分析.
主要方法:
- 开发了文本控制的Motion Mamba (TM-Mamba),这是一个具有线性内存成本的统一模型.
- 集成的时间全球上下文,语言查询控制和空间图形拓.
- 引入了以文本控制的选择机制,以实现动态的全球信息整合.
- 通过空间图形拓学的关系嵌入来增强模型意识.
主要成果:
- TM-Mamba 在基于文本的人类运动接地方面表现出有效性.
- 该模型有效地处理具有线性内存复杂性的长无修剪序列.
- 在新推出的 BABEL-Grounding 数据集上取得了强的表现.
结论:
- TM-Mamba提供了一种高效和有效的解决方案,用于基于文本的人类运动接地.
- 提出的方法克服了长序分析的变压器模型的局限性.
- 贝贝尔-Grounding数据集有助于在这个领域的未来研究.
更多相关视频
10:52Simulation of Human-induced Vibrations Based on the Characterized In-field Pedestrian Behavior
Published on: April 13, 2016
9.1K
11:06A Human-machine-interface Integrating Low-cost Sensors with a Neuromuscular Electrical Stimulation System for Post-stroke Balance Rehabilitation
Published on: April 12, 2016
10.8K
相关概念视频
Relative Motion Analysis using Rotating Axes
870
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
870
Relative Motion Analysis using Rotating Axes-Problem Solving
690
Consider a crane whose telescopic boom rotates with an angular velocity of 0.04 rad/s and angular acceleration of 0.02 rad/s2. Along with the rotation, the boom also extends linearly with a uniform speed of 5 m/s. The extension of the boom is measured at point D, which is measured with respect to the fixed point C on the other end of the boom. For the given instant, the distance between points C and D is 60 meters.
Here, in order to determine the magnitude of velocity and acceleration for point...
Here, in order to determine the magnitude of velocity and acceleration for point...
690
Absolute Motion Analysis- General Plane Motion
522
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
522
Relative Motion Analysis - Velocity
679
A stroke engine has a slider-crank mechanism that converts rotational motion from the crank into linear motion of the slider or vice versa. This mechanism consists of three main parts: the crank, the connecting rod, and the slider.
When an external force is exerted, it sets the crank into a rotational movement. This, in turn, instigates the motion of the connecting rod, leading to what is referred to as a general plane motion. This process involves two key points - point A on the connecting rod...
When an external force is exerted, it sets the crank into a rotational movement. This, in turn, instigates the motion of the connecting rod, leading to what is referred to as a general plane motion. This process involves two key points - point A on the connecting rod...
679
Relative Motion Analysis - Acceleration
798
A slider-crank mechanism converts rotational motion from the crank into linear motion of the slider or vice versa. This mechanism consists of three main parts: the crank, the connecting rod, and the slider. The movement of the slider-crank is an example of general plane motion as the fluctuating angle between the crank and the connecting rod. Consider a segment AB where point A is at the end of the slider and point B is on the diametrically opposite end to point A, on a crack. The variance in...
798
Relative Motion Analysis using Rotating Axes - Acceleration
743
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame. The absolute velocity of point B is determined by adding the absolute velocity of point A, the relative velocity of point B in the rotating frame, and the effects caused by the angular velocity within the rotating frame.
Time differentiation is...
Time differentiation is...
743
