SMC++:无监督视频语义压缩的蒙面学习
IEEE transactions on pattern analysis and machine intelligence
|October 23, 2025
概括
本研究介绍了一种新的视频压缩框架,使用掩面视频建模 (MVM) 来保存视频语义. 语义挖掘然后压缩 (SMC) 模型通过将语义内容优先于视觉细节来显著改善视频分析任务.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 视频处理 视频处理
背景情况:
- 传统的视频压缩优先考虑人类的视觉感知,往往导致显著的语义损失.
- 这种语义损失阻碍了下游视频分析任务的执行.
- 现有的方法努力平衡压缩效率与语义保存.
研究的目的:
- 提出一个有效保存语义信息的视频压缩框架.
- 开发一种自我监督的方法,共同挖掘和压缩视频语义.
- 在压缩过程中减轻非语义信息和语义噪声的编码.
主要方法:
- 使用掩面视频建模 (MVM) 通过掩面补丁预测来学习可概括的语义.
- 在MVM代币空间中引入非语义的明确规范化.
- 实例化框架作为语义挖掘然后压缩 (SMC) 和先进的SMC++模型.
- 在SMC++中整合了面具运动预测和基于变压器的压缩模块.
- 使用紧的蓝图语义表示来对齐异质特征.
主要成果:
- 拟议的SMC和SMC++模型与传统,可学习和感知质量导向编解码器相比,表现出更高的性能.
- 在三个视频分析任务和七个不同的数据集中观察到显著的改善.
- 该框架有效地保留了关键的语义内容,同时实现了高效的压缩.
结论:
- 基于MVM的压缩框架为语义维护视频压缩提供了一个新的范式.
- 通过保持语义完整性,SMC和SMC++提供了一个强大的解决方案来增强视频分析.
- 这种方法通过解决以感知质量为中心的压缩方法的局限性来推动该领域的发展.
相关概念视频
Masking and Demasking Agents
3.4K
EDTA titrations may necessitate masking and demasking agents to temporarily protect a particular metal ion in a mixture from the EDTA reaction. These agents facilitate the sequential analysis of the metal ions by forming stable complexes with some—but not all—metal ions during certain steps.
There are many masking agents, such as cyanide, fluoride, triethanolamine, thiourea, and 2,3-bis(sulfanyl)propan-1-ol (formerly 2,3-dimercapto-1-propanol), with the masking agent chosen based on...
There are many masking agents, such as cyanide, fluoride, triethanolamine, thiourea, and 2,3-bis(sulfanyl)propan-1-ol (formerly 2,3-dimercapto-1-propanol), with the masking agent chosen based on...
3.4K
Uniform Depth Channel Flow: Problem Solving
426
To calculate the flow rate for a trapezoidal channel, first, identify the bottom width, side slope, and flow depth of the channel. The cross-sectional area (A) corresponding to the depth of flow (y), channel bottom width (B), and side slope (θ) is determined by:Next, calculate the wetted perimeter, which includes the bottom width and the sloped side lengths in contact with the water. Using the values of the cross-sectional area and the wetted perimeter, determine the hydraulic radius by...
426
Stereotype Content Model
15.3K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
15.3K
Extraction: Advanced Methods
1.1K
Metal ions can be separated from one another by complexation with organic ligands–the chelating agent– to form uncharged chelates. Here, the chelating agent must contain hydrophobic groups and behave as a weak acid, losing a proton to bind with the metal. Since most organic ligands used in this process are insoluble or undergo oxidation in the aqueous phase, the chelating agent is initially added to the organic phase and extracted into the aqueous phase. The metal-ligand complex is...
1.1K
Reducing Line Loss
355
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss in...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss in...
355
Introduction to Learning
931
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
931
