超低比特率面部视频压缩基于从3D关键点转换到2D运动图的转换
概括
本研究介绍了一种使用3D关键点和2D运动的新面部视频压缩方法. 它实现了高质量的重建和可靠的视频通信的视觉一致性,即使在2 kbps.
科学领域:
- 计算机视觉 计算机视觉
- 信号处理 信号处理
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 面部视频压缩对于视频聊天和远程教育等在线应用至关重要.
- 现有的方法在3D面部运动和2D视觉表现之间的不一致性中扎.
- 由于结构信息,深度生成模型提供了高质量的面部视频压缩的潜力.
研究的目的:
- 提出面部视频压缩的新型生成方法,确保感知补偿和视觉一致性.
- 为了解决当前方法在调整3D物理运动与2D视觉演变中的局限性.
- 为了实现面部为中心的视频的优越压缩性能.
主要方法:
- 介绍了一个基于3D-Keypoint和2D-Motion的生成方法 (FVC-3K2M).
- 用全球和本地3D关键点来编码灵活性和运动精度来表征时间面部视频演变.
- 开发了一种级联运动转换机制,以将3D关键点转换为2D密集运动,以实现现实的重建.
- 实施了适应性参考框架选择方案,以处理各种时间运动.
主要成果:
- 拟议的FVC-3K2M方案可在极低带宽 (例如2 kbps) 上实现可靠的视频通信.
- 与最先进的视频编码标准和最新的面部视频压缩方法相比,实现了优越的压缩性能.
- 演示了增强的感知补偿和运动和重建之间的视觉一致性.
结论:
- FVC-3K2M方法有效地克服了生成面部视频压缩中的3D到2D运动不一致性.
- 该技术为面部为中心的视频提供了压缩效率和视觉质量的显著改进.
- 这种方法有望在带宽受限制的环境中实现高质量的视频通信.
相关概念视频
01:25Relative Motion Analysis using Rotating Axes
441
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
441
01:24Absolute Motion Analysis- General Plane Motion
199
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
199
01:24Relative Motion Analysis - Velocity
334
A stroke engine has a slider-crank mechanism that converts rotational motion from the crank into linear motion of the slider or vice versa. This mechanism consists of three main parts: the crank, the connecting rod, and the slider.
When an external force is exerted, it sets the crank into a rotational movement. This, in turn, instigates the motion of the connecting rod, leading to what is referred to as a general plane motion. This process involves two key points - point A on the connecting rod...
When an external force is exerted, it sets the crank into a rotational movement. This, in turn, instigates the motion of the connecting rod, leading to what is referred to as a general plane motion. This process involves two key points - point A on the connecting rod...
334
01:20Downsampling
121
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
121
01:29Relative Motion Analysis using Rotating Axes-Problem Solving
382
Consider a crane whose telescopic boom rotates with an angular velocity of 0.04 rad/s and angular acceleration of 0.02 rad/s2. Along with the rotation, the boom also extends linearly with a uniform speed of 5 m/s. The extension of the boom is measured at point D, which is measured with respect to the fixed point C on the other end of the boom. For the given instant, the distance between points C and D is 60 meters.
Here, in order to determine the magnitude of velocity and acceleration for point...
Here, in order to determine the magnitude of velocity and acceleration for point...
382
01:10Relative Motion Analysis - Acceleration
320
A slider-crank mechanism converts rotational motion from the crank into linear motion of the slider or vice versa. This mechanism consists of three main parts: the crank, the connecting rod, and the slider. The movement of the slider-crank is an example of general plane motion as the fluctuating angle between the crank and the connecting rod. Consider a segment AB where point A is at the end of the slider and point B is on the diametrically opposite end to point A, on a crack. The variance in...
320