空间和频率信息融合变压器用于超分辨率图像
概括
本研究介绍了空间和频率信息融合变压器 (SFFT),用于增强单图像超分辨率 (SISR). 通过整合空间和频率数据,SFFT模型显著改善了图像重建,优于现有的方法.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 图像处理 图像处理
背景情况:
- 变压器模型显示单图像超分辨率 (SISR) 的承诺.
- 目前的基于变压器的方法使用非重叠的窗口,限制受体场和全球信息捕获.
- 捕获远距离依赖关系至关重要,特别是在早期的网络层中,以便有效的图像重建.
研究的目的:
- 为SISR开发一种基于变压器的新型模型,具有扩展的受体场.
- 通过有效整合空间和频率域信息来增强图像重建.
- 提高网络捕获全球图像特征和远距离依赖性的能力.
主要方法:
- 提出了空间和频率信息融合变压器 (SFFT) 模型.
- SFFT集成了空间和频域信息,以捕获本地和全球图像特征.
- 引入了重叠交叉注意区块 (OCAB) 以改善窗口之间的像素传输.
- 在训练期间内置快速里埃转换 (FFT) 损失以优化模块性能.
主要成果:
- 在对基准数据集的定量和定性评估中,SFFT模型表现出卓越的性能.
- 在Manga109数据集上获得了32.67dB的PSNR得分,比SwinIR高出0.64dB,比HAT高出0.19dB.
- 拟议的方法有效地利用全球信息,并激活更多的像素,以改善图像重建.
结论:
- 在单图像超分辨率方面,SFFT模型提供了显著的进步.
- 空间和频率信息的融合,加上OCAB和FFT损失,提高了重建的准确性.
- 拟议的方法为SISR任务提供了最先进的性能.
相关概念视频
Super-resolution Fluorescence Microscopy
6.8K
Super-resolution fluorescence microscopy (SRFM) provides a better resolution than conventional fluorescence microscopy by reducing the point spread function (PSF). PSF is the light intensity distribution from a point that causes it to appear blurred. Due to PSF, each fluorescing point appears bigger than its actual size, and it is the PSF interference of nearby fluorophores that causes the blurred image. Various approaches to achieving higher resolution through SRFM have recently been...
6.8K
Upsampling
180
Managing signal sampling rates is essential in digital signal processing to maintain signal integrity. A decimated signal, characterized by a reduced frequency range due to its lower sampling rate, can be upsampled by inserting zeros between each sample. This upsampling process expands the original spectrum and introduces repeated spectral replicas at intervals dictated by the new Nyquist frequency. To refine this zero-inserted sequence, it is passed through a lowpass filter with a cutoff...
180
Continuous -time Fourier Transform
256
The Fourier series is instrumental in representing periodic functions, offering a powerful method to decompose such functions into a sum of sinusoids. This technique, however, necessitates modification when applied to nonperiodic functions. Consider a pulse-train waveform consisting of a series of rectangular pulses. When these pulses have a finite period, they can be accurately represented by a Fourier series. Yet, as the period approaches infinity, resulting in a single, isolated pulse, the...
256
Reconstruction of Signal using Interpolation
154
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
154


