RetinaViT:在线视频流的高效视觉支柱
Tomoyuki Suzuki1, Yoshimitsu Aoki1
1Department of Electronics and Electrical Engineering, Faculty of Science and Technology, Keio University, 3-14-1, Hiyoshi, Kohoku-ku, Yokohama 223-8522, Kanagawa, Japan.
Sensors (Basel, Switzerland)
|September 14, 2024
概括
RetinaViT通过高效地提取级视觉特征来增强在线视频的理解. 这种方法显著加快了诸如动作识别等任务的速度,提高了准确性和效率.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 在线视频理解对于许多应用程序至关重要.
- 级视觉特征提取是视频处理中的一个主要瓶.
- 现有的方法难以满足实时推断速度要求.
研究的目的:
- 提出RetinaViT,一种有效的在线视频理解方法.
- 为了提高级视觉特征提取的速度和准确性.
- 为了提高在线视频理解任务的整体效率.
主要方法:
- RetinaViT使用近似的变压器块与事件令牌作为查询.
- 它重复使用先前处理的代币,并将密钥/值限制在空间社区.
- 模型参数在训练期间通过多步黑子优化进行调整.
主要成果:
- 在各种任务中,RetinaViT显著提高了速度/精度的权衡.
- 对于动作识别,它将推断时间缩短至61.9% (CPU) 和50.8% (GPU).
- 与基线模型相比,准确性保持或略有改善.
结论:
- RetinaViT为在线视频理解提供了显著的效率提升.
- 该方法有效地解决了框架级特征提取的瓶.
- RetinaViT展示了对现实世界的视频分析应用程序的实际好处.
相关概念视频
The Retina
68.0K
The retina is a layer of nervous tissue at the back of the eye that transduces light into neural signals. This process, called phototransduction, is carried out by rod and cone photoreceptor cells in the back of the retina.
68.0K
Vision
53.1K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.1K
Reducing Line Loss
150
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
150
Visual System
557
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
557
Depth Perception and Spatial Vision
609
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
609
Uniform Depth Channel Flow
63
Uniform depth channel flow keeps fluid depth consistent along channels such as irrigation canals. In natural channels, such as rivers, approximate uniform flow is often assumed. This condition occurs when the channel’s bottom slope matches the energy slope, balancing potential energy lost from gravity with head loss due to shear stress. This balance prevents depth changes along the channel length, resulting in a steady, uniform flow.Uniform flow in open channels with a constant cross-section...
63


