快照压缩成像的实时解码使用张量器FISTA-Net.
概括
我们开发了Tensor FISTA-Net,这是用于快照压缩成像 (SCI) 摄像机的深度学习解码器. 这种新的方法显著提高了高速视频和超频谱图像的解码速度和质量,使实时应用成为可能.
科学领域:
- 光学和光子学 在光学和光子学.
- 计算机视觉 计算机视觉
- 机器学习 机器学习
背景情况:
- 快照压缩成像 (SCI) 将高速视频或超光谱图像压缩到更少的测量中.
- 解码这些是计算密集的,由于质量低或处理时间长,限制了当前算法用于实时使用.
研究的目的:
- 为SCI摄像头开发一种新,高效,高质量的解码器,适合实时应用.
- 利用深度神经网络 (DNN) 和张量运算来加速和准确的图像重建.
主要方法:
- 提议了Tensor FISTA-Net,这是一个深度神经网络,将卷积层结合起来,用于稀疏表示和张量运算以提高效率.
- 在使用SCI摄像机的物理模型生成的丰富合成数据上训练解码器,以优化解码性能.
- 在真实的SCI数据集 (子和轮) 上对解码器进行了评估,用于视频和高光谱成像.
主要成果:
- 与最先进的方法相比,在峰值信号噪声比率 (PSNR) 中取得了显著的改进:视频为0.79-2.84dB,高频谱图像为2.61-4.43dB.
- 通过更清晰,更详细的重建图像,证明了卓越的视觉质量.
- 实现实时解码速度:视频为每秒45 (fps),超频谱数据为每秒70.
- 经过训练的模型具有很小的内存空间 (12 MB),适合物联网 (IoT) 设备.
结论:
- Tensor FISTA-Net为实时SCI解码提供了一个实用的解决方案,克服了现有算法的局限性.
- 深度学习方法有效地平衡了解码质量,速度和计算资源.
- 该方法的效率和低内存要求使其非常适合用于SCI中的实时和边缘计算应用.
相关概念视频
Computed Tomography
4.6K
Tomography refers to imaging by sections. Computed tomography (CT) is a non-invasive imaging technique that uses computers to analyze several cross-sectional X-rays to reveal minute details about structures in the body.
The technique was invented in the 1970s and is based on the principle that as X-rays pass through the body, they are absorbed or reflected at different levels. In the technique, a patient lies on a motorized platform while a computerized axial tomography (CAT) scanner rotates...
The technique was invented in the 1970s and is based on the principle that as X-rays pass through the body, they are absorbed or reflected at different levels. In the technique, a patient lies on a motorized platform while a computerized axial tomography (CAT) scanner rotates...
4.6K
Fast Fourier Transform
406
The Fast Fourier Transform (FFT) is a computational algorithm designed to compute the Discrete Fourier Transform (DFT) efficiently. By breaking down the calculations into smaller, manageable sections, the FFT significantly reduces the computational complexity involved. Direct computation of an N-point DFT requires N2 complex multiplications, whereas the FFT algorithm needs only (N/2)log2N multiplications, offering a much faster performance.
The computational efficiency of the FFT becomes...
The computational efficiency of the FFT becomes...
406
Reconstruction of Signal using Interpolation
256
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
256
Imaging Studies III: Computed Tomography
30
DefinitionComputed Tomography (CT) of the genitourinary (GU) tract is a non-invasive imaging modality that utilizes X-rays and computer processing to generate detailed cross-sectional images of the urinary system, encompassing the kidneys, ureters, bladder, and adjacent structures such as the adrenal glands.PurposeCT scans of the GU tract serve several diagnostic and therapeutic purposes, including:Diagnosis of Urinary Tract Diseases: Detects kidney stones, tumors, cysts, and congenital...
30


