用于实时视频超分辨率重建的GPU芯片上的网络设计
Zhiyong Peng1, Jiang Du1, Yulong Qiao2
1School of Optoelectronic Engineering, Guilin University of Electronic Technology, Guilin 541004, China.
Micromachines
|May 27, 2023
概括
本研究介绍了使用GPU并行加速的更快的深度学习视频超分辨率 (SR) 算法. 新方法实现了实时性能,提高了SR质量,超过了现有的算法.
科学领域:
- 计算机视觉 计算机视觉
- 深度学习 (Deep Learning) 是一种深度学习.
- 图像处理 图像处理
背景情况:
- 深度学习在视频超分辨率 (SR) 质量方面表现出色,但受到高资源需求和低实时性能的影响.
- 传统的SR算法缺乏深度学习方法的质量.
- 实时视频SR需要高效的算法和硬件加速.
研究的目的:
- 解决基于深度学习的视频超分辨率 (SR) 的速度限制.
- 通过优化深度学习和GPU并行加速实现实时视频SR.
- 开发一个计算效率高的SR算法,适合GPU实现.
主要方法:
- 开发了一种新的视频SR算法,将深度学习网络与查找表 (LUT) 结合起来,以实现增强的GPU并行加速.
- 实施了三个关键的GPU优化策略:存储访问,条件分支和线程,以提高计算效率.
- 将优化的算法集成到RTX 3090 GPU上实现的芯片上的网络架构中.
主要成果:
- 拟议的GPU网络芯片算法实现了实时性能,以42FPS (540x540分辨率) 处理真实视频.
- 这种新方法显示出更高的效率,速度是原来的SR-LUT-S快速方法的9.1倍.
- 实验结果显示SR质量有所改善,平均PSNR值比SR-LUT-V高0.61dB,比SR-LUT-S高0.24dB.
结论:
- 深度学习SR算法和GPU并行加速的协作设计有效地解决了实时性能问题.
- 与现有方法相比,拟议的算法提供了显著的加快速度,同时保持或提高了超分辨率质量.
- 这种方法可以在资源有限的硬件上实现实用的实时视频超分辨率应用.
相关概念视频
Upsampling
267
Managing signal sampling rates is essential in digital signal processing to maintain signal integrity. A decimated signal, characterized by a reduced frequency range due to its lower sampling rate, can be upsampled by inserting zeros between each sample. This upsampling process expands the original spectrum and introduces repeated spectral replicas at intervals dictated by the new Nyquist frequency. To refine this zero-inserted sequence, it is passed through a lowpass filter with a cutoff...
267
Reconstruction of Signal using Interpolation
256
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
256
Downsampling
199
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
199


