细分视频的动作组件:长期无监督模型
IEEE transactions on pattern analysis and machine intelligence
|September 9, 2025
概括
本研究介绍了一种无监督的基于变压器的模型,用于稳定的视频运动细分. 它通过光学流场和新的损失函数准确地在整个视频序列中分割连贯的运动.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 人类的视觉感知在分析视频中的运动方面表现出色.
- 目前的方法往往在稳定和连贯的运动细分过程中难以实现.
- 需要对运动细分进行无监督的方法.
研究的目的:
- 开发一种新的,无监督的,长期的时空模型,用于视频中连贯的运动细分.
- 为了在单次传递中处理整个视频序列以进行运动细分.
- 为了提高运动细分结果的时间一致性.
主要方法:
- 提出了一个基于变压器的网络架构.
- 该模型使用连续光流 (OF) 场作为输入.
- 使用来自证据下界 (ELBO) 框架的损失函数,将流量重建与时间一致性规范化结合起来.
主要成果:
- 该模型在四个视频对象细分 (VOS) 基准上取得了具有竞争力的定量结果.
- 它在整个视频序列上展示了有效的运动细分.
- 视觉结果突出显著改善时间一致性.
结论:
- 拟议的无监督模型为连贯的运动细分提供了一个强大的解决方案.
- 时空运动模型和ELBO衍生损失的新组合增强了时间稳定性.
- 这种方法推进了无监督视频分析和运动理解领域.
相关概念视频
Absolute Motion Analysis- General Plane Motion
540
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
540
Relative Motion Analysis using Rotating Axes
883
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
883
Relative Motion Analysis using Rotating Axes-Problem Solving
705
Consider a crane whose telescopic boom rotates with an angular velocity of 0.04 rad/s and angular acceleration of 0.02 rad/s2. Along with the rotation, the boom also extends linearly with a uniform speed of 5 m/s. The extension of the boom is measured at point D, which is measured with respect to the fixed point C on the other end of the boom. For the given instant, the distance between points C and D is 60 meters.
Here, in order to determine the magnitude of velocity and acceleration for point...
Here, in order to determine the magnitude of velocity and acceleration for point...
705
Curvilinear Motion: Rectangular Components
1.1K
Curvilinear motion characterizes the movement of a particle or object along a curved path, notably evident when envisioning a car navigating a winding road. If the car starts at point A, its position vector is established within a fixed frame of reference, where the ratio of the position vector to its magnitude signifies the unit vector pointing in the position vector's direction.
As the car advances, its position evolves over time. Quantifying the car's velocity involves computing the...
As the car advances, its position evolves over time. Quantifying the car's velocity involves computing the...
1.1K

