Image Super-Resolution via Efficient Transformer Embedding Frequency Decomposition With Restart.

Area of Science:

• Computer Vision
• Deep Learning
• Image Processing

Background:

• Transformer backbones outperform convolutional networks in computer vision.
• Local attention in transformers is used for low-level image processing due to its linear complexity.
• Limited receptive fields in local attention hinder performance.

Purpose of the Study:

• To propose a transformer-based single image super-resolution (SISR) model that addresses the limitations of local attention.
• To introduce dynamic frequency decomposition within a local transformer architecture.
• To improve receptive field size and reduce computational complexity in SISR.

Main Methods:

• Proposed a novel transformer-based SISR model incorporating dynamic frequency decomposition (FDRTran layer).
• Employed intra-scale attention and inter-scale interaction for continuous updating and re-assignment of frequency components.
• Introduced a restart mechanism for feature fusion and re-decomposition to avoid representation saturation.

Main Results:

• The FDRTran layer decreases both FLOPs and parameters compared to standard local transformers.
• The proposed FDRNet achieves state-of-the-art performance on six synthetic and real-world datasets.
• FDRNet is the first transformer backbone for SISR to incorporate Octave design principles.

Conclusions:

• The proposed dynamic frequency decomposition effectively enhances transformer-based SISR models.
• FDRNet offers a computationally efficient and high-performing solution for single image super-resolution.
• The model's architecture and mechanisms provide a novel approach to frequency representation in deep learning for image restoration.