Spatial and frequency information fusion transformer for image super-resolution

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Transformer models show promise for single image super-resolution (SISR).
• Current Transformer-based methods use non-overlapping windows, limiting receptive fields and global information capture.
• Capturing long-distance dependencies is crucial, especially in early network layers for effective image reconstruction.

Purpose of the Study:

• To develop a novel Transformer-based model for SISR with an expanded receptive field.
• To enhance image reconstruction by effectively integrating spatial and frequency domain information.
• To improve the network's ability to capture global image features and long-distance dependencies.

Main Methods:

• Proposed the Spatial and Frequency Information Fusion Transformer (SFFT) model.
• SFFT integrates spatial and frequency domain information to capture local and global image features.
• Introduced the overlapping cross-attention block (OCAB) for improved pixel transmission between windows.
• Incorporated Fast Fourier Transform (FFT) loss during training to optimize module performance.

Main Results:

• The SFFT model demonstrated superior performance in quantitative and qualitative evaluations on benchmark datasets.
• Achieved a PSNR score of 32.67 dB on the Manga109 dataset, outperforming SwinIR by 0.64 dB and HAT by 0.19 dB.
• The proposed method effectively leverages global information and activates more pixels for improved image reconstruction.

Conclusions:

• The SFFT model offers a significant advancement in single image super-resolution.
• The fusion of spatial and frequency information, coupled with OCAB and FFT loss, enhances reconstruction accuracy.
• The proposed approach provides state-of-the-art performance for SISR tasks.