通过:补丁自动跳过方案,以实现高效的设备内视频感知
IEEE transactions on pattern analysis and machine intelligence
|January 8, 2024
概括
补丁自动跳过方案 (PASS) 通过在补丁级别智能跳过冗余计算来加速边缘设备上的视频感知. 这种独立于任务的方法可以提高效率,而不会牺牲准确性.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 在资源有限的边缘设备上实时视频感知面临着精度和硬件开销方面的挑战.
- 当前的方法通常需要特定任务的硬件或预先优化的模型,限制了通用性.
研究的目的:
- 开发一种一般的,独立于任务的方法来加速边缘设备上的视频感知任务.
- 引入补丁自动跳过方案 (PASS) 以通过选择性跳过冗余补丁进行高效的计算.
主要方法:
- PASS利用卷积层中的可学习门来识别和跳过非关键图像补丁.
- 一种新的自我监督程序通过从序列中学习对比表示来优化这些门.
- 预先训练的门可以作为各种神经骨干的插电模块.
主要成果:
- 通过能够显著加速各种基于视频的下游任务.
- 在NVIDIA Jetson Nano上,在3D姿势估计 (MobileHumanPose) 中达到9.43倍的加速度,在多重对象跟踪 (FairMOT) 中达到12.19倍.
- 证明了有效的加速,而不会影响模型的准确性.
结论:
- PASS提供了一种通用和有效的解决方案,用于加速边缘设备上的视频感知.
- 独立于任务的性质和插件模块使PASS能够适应各种应用.
- 这种方法解决了对实时边缘AI高效计算的关键需求.
更多相关视频
08:13SwarmSight: Real-time Tracking of Insect Antenna Movements and Proboscis Extension Reflex Using a Common Preparation and Conventional Hardware
Published on: December 25, 2017
8.2K
07:52Automated Rat Single-Pellet Reaching with 3-Dimensional Reconstruction of Paw and Digit Trajectories
Published on: July 10, 2019
14.2K
相关概念视频
Downsampling
158
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
158
Fixed Action Patterns
16.0K
A fixed action pattern (FAP) is a specific, hard-wired sequence of behaviors that occurs in response to an external stimulus, called a sign stimulus. The behavior is “fixed” because it is essentially unchangeable—proceeding similarly across individuals of a species every time it occurs.
16.0K
Upsampling
238
Managing signal sampling rates is essential in digital signal processing to maintain signal integrity. A decimated signal, characterized by a reduced frequency range due to its lower sampling rate, can be upsampled by inserting zeros between each sample. This upsampling process expands the original spectrum and introduces repeated spectral replicas at intervals dictated by the new Nyquist frequency. To refine this zero-inserted sequence, it is passed through a lowpass filter with a cutoff...
238
Parallel Processing
152
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
152
Depth Perception and Spatial Vision
657
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
657
Reducing Line Loss
154
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
154
