MADiff:在自我中心的视频上预测手部轨迹的运动感知马巴扩散模型
IEEE transactions on pattern analysis and machine intelligence
|November 17, 2025
概括
这项研究引入了MADiff,这是一种从自我中心视频中预测未来手动的新方法. MADiff使用扩散模型和感知运动的Mamba来改善对人工智能的理解.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器人技术 机器人技术 机器人技术
背景情况:
- 自我中心的视觉对于体现的人工智能至关重要,手的轨迹预测对于理解人类运动至关重要.
- 挑战包括从自我中心的视频中推断高层次的意图和时间因果关系,特别是在相机运动和缺乏负担能力标签的情况下.
研究的目的:
- 提出MADiff,一种使用扩散模型的新手手轨迹预测方法.
- 通过整合自我运动和利用语义理解的基础模型来解决自我中心视频分析的挑战.
主要方法:
- 使用扩散模型来预测未来的手指路标.
- 引入了运动感知Mamba,具有运动驱动选择性扫描 (MDSS) 功能,用于隐性空间的无声化,集成相机佩戴者的自我运动.
- 采用一个融合视觉和语言特征的基础模型,以捕捉高级场景语义,而无需明确的负担能力监督.
主要成果:
- 在手的轨迹预测中,MADiff表现出与最先进的基线相当的性能.
- 在五个公共数据集的实验中,使用现有和新的评估指标验证了该方法的有效性.
- 该方法成功地预测了相当准确的手的轨迹,改进了当前的方法.
结论:
- 在自我中心的视频中,MADiff提供了一个强大的解决方案来预测手的轨迹.
- 该方法通过提高对人类意图和行为的理解来增强体现的人工智能研究.
- 未来的工作可以建立在这种方法上,用于先进的机器人技术和扩展现实应用.
相关概念视频
Absolute Motion Analysis- General Plane Motion
510
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
510
Relative Motion Analysis using Rotating Axes
865
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
865
Relative Motion Analysis - Velocity
676
A stroke engine has a slider-crank mechanism that converts rotational motion from the crank into linear motion of the slider or vice versa. This mechanism consists of three main parts: the crank, the connecting rod, and the slider.
When an external force is exerted, it sets the crank into a rotational movement. This, in turn, instigates the motion of the connecting rod, leading to what is referred to as a general plane motion. This process involves two key points - point A on the connecting rod...
When an external force is exerted, it sets the crank into a rotational movement. This, in turn, instigates the motion of the connecting rod, leading to what is referred to as a general plane motion. This process involves two key points - point A on the connecting rod...
676
Relative Motion Analysis - Acceleration
785
A slider-crank mechanism converts rotational motion from the crank into linear motion of the slider or vice versa. This mechanism consists of three main parts: the crank, the connecting rod, and the slider. The movement of the slider-crank is an example of general plane motion as the fluctuating angle between the crank and the connecting rod. Consider a segment AB where point A is at the end of the slider and point B is on the diametrically opposite end to point A, on a crack. The variance in...
785
Relative Motion Analysis using Rotating Axes-Problem Solving
685
Consider a crane whose telescopic boom rotates with an angular velocity of 0.04 rad/s and angular acceleration of 0.02 rad/s2. Along with the rotation, the boom also extends linearly with a uniform speed of 5 m/s. The extension of the boom is measured at point D, which is measured with respect to the fixed point C on the other end of the boom. For the given instant, the distance between points C and D is 60 meters.
Here, in order to determine the magnitude of velocity and acceleration for point...
Here, in order to determine the magnitude of velocity and acceleration for point...
685
Relative Motion Analysis using Rotating Axes - Acceleration
733
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame. The absolute velocity of point B is determined by adding the absolute velocity of point A, the relative velocity of point B in the rotating frame, and the effects caused by the angular velocity within the rotating frame.
Time differentiation is...
Time differentiation is...
733


