VNVC:一个多功能的神经视频编码框架,用于高效的人机视觉
IEEE transactions on pattern analysis and machine intelligence
|January 22, 2024
概括
本研究介绍了一种多功能神经视频编码 (VNVC) 框架. 它可以直接增强和分析压缩视频,提高人类和机器视觉任务的效率.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 信号处理 信号处理
背景情况:
- 数字视频压缩对于传输至关重要,但对像素的解码对于人类显示和机器分析是必要的.
- 目前的方法需要将压缩视频解码为像素进行增强和分析,这是计算密集的.
- 直接处理压缩的视频表示提供了潜在的效率增长.
研究的目的:
- 提出一个多功能的神经视频编码 (VNVC) 框架,支持视频重建和直接增强/分析从紧的表示.
- 开发基于功能的压缩循环,以实现高效的视频压缩和多功能下游应用.
- 从压缩数据直接实现人类视觉 (重建) 和机器视觉 (增强/分析) 的高效处理.
主要方法:
- 引入了一个多功能的神经视频编码 (VNVC) 框架,具有基于功能的压缩循环.
- 将编码为紧的表示,在重建之前将其解码为中间特征.
- 利用基于特征的时间上下文挖掘和跨域的运动编码器-解码器进行框架间压缩.
- 直接将中间功能输入网络,用于视频重建,增强和分析.
主要成果:
- 实现了用于视频重建的高压缩效率.
- 在视频增强和分析方面表现出令人满意的任务表现.
- 与传统方法相比,展示了较低的计算复杂性.
- 验证了中间特征对多功能视觉任务的有效性.
结论:
- 拟议的VNVC框架为视频压缩提供了一种多功能方法,支持重建和直接分析.
- 基于特征的压缩和中间特征的直接处理导致了显著的效率增长.
- 这种框架对人类和机器视觉应用都有效,减少了计算负载.
相关概念视频
Vision
53.4K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.4K
Neural Circuits
1.2K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.2K
Parallel Processing
152
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
152
Machines
271
Machines are complex structures consisting of movable, pin-connected multi-force members that work together to transmit forces. One example of a machine is the cutting plier, which is used to cut wires by applying forces to its handles. When equal and opposite forces are exerted on the handles of the cutting plier, they cause the cutting edges to come together and apply equal and opposite reaction forces on the wire, which are greater than the applied forces.
A free-body diagram of the...
A free-body diagram of the...
271
Visual System
585
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
585
Neural Regulation
39.4K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
39.4K


