Lightweight Single Image Super-Resolution via Efficient Mixture of Transformers and Convolutional Networks

Area of Science:

• Computer Vision
• Artificial Intelligence
• Deep Learning

Background:

• Single Image Super-Resolution (SISR) is crucial for enhancing image quality.
• Existing methods often struggle to balance local details and global context effectively.
• Lightweight and high-performance networks are needed for practical SISR applications.

Purpose of the Study:

• To propose a novel network architecture, the Local Global Union Network (LGUN), for efficient and high-performance Single Image Super-Resolution (SISR).
• To leverage the complementary strengths of Transformers and Convolutional Networks in a unified framework.
• To demonstrate the effectiveness of LGUN on both natural and satellite image datasets.

Main Methods:

• The Local Global Union Network (LGUN) integrates Transformers for global context and Convolutional Networks for local features.
• Multi-order Local Hierarchical Attention (MLHA) is used in shallow layers to encode local spatial information.
• Dynamic Global Sparse Attention (DGSA) with Multi-stage Token Selection (MTS) is employed in deeper layers for global context modeling.

Main Results:

• LGUN effectively combines global context interaction and local feature extraction.
• The network demonstrates superior performance compared to existing SISR methods.
• Experiments were conducted on both natural and satellite image datasets from optical and satellite sensors.

Conclusions:

• LGUN offers a lightweight yet high-performance solution for Single Image Super-Resolution.
• The proposed architecture successfully merges the benefits of Transformers and Convolutional Networks.
• LGUN shows significant potential for various image super-resolution tasks across different sensor types.