StarIR: Convolutional Image Restoration With Spatial-Frequency Fusion.

Area of Science:

• Computer Vision
• Deep Learning
• Image Processing

Background:

• Vision Transformers (ViTs) offer large receptive fields for image restoration but are computationally complex for high resolutions.
• Convolutional Neural Networks (CNNs) are efficient but limited by local receptive fields, hindering long-range dependency capture.

Purpose of the Study:

• To develop an image restoration model that combines the efficiency of CNNs with the long-range dependency capture of Transformers.
• To introduce a novel architecture, StarIR, addressing the limitations of existing methods for high-resolution image restoration.

Main Methods:

• Proposed StarIR, a dual-domain representation learning framework processing spatial and frequency domains.
• Introduced a Star operation for high-dimensional feature fusion via element-wise multiplication, enhancing representational capacity without increasing network size.
• Integrated a channel attention unit for global feature modeling and improved channel-wise interactions.

Main Results:

• Achieved state-of-the-art performance on 21 datasets across six single-degradation image restoration tasks.
• Demonstrated superior performance against leading algorithms in all-in-one and composite-degradation settings.
• Showcased successful application in ultra-high-definition (UHD) imaging, remote sensing, medical imaging, and underwater image enhancement.

Conclusions:

• StarIR offers an efficient yet powerful solution for image restoration, outperforming existing methods.
• The dual-domain approach and Star operation effectively enhance feature representation and capture long-range dependencies.
• StarIR demonstrates broad applicability and robustness across diverse image restoration challenges and domains.