TC3Net: Transformer and Convolution Coupled Contrastive Network for Single Image Super-Resolution.

Area of Science:

• Computer Vision
• Deep Learning
• Image Processing

Background:

• Convolutional Neural Networks (CNNs) and Transformers excel at nonlinear feature extraction for Single Image Super-Resolution (SISR).
• CNNs have limitations in kernel-image interaction, while Transformers face quadratic computational complexity with increasing resolution.
• Existing methods struggle to balance performance and model size in SISR.

Purpose of the Study:

• To propose a novel unified framework, the Transformer and Convolution Coupled Contrastive Network (TC3Net), for SISR.
• To integrate the complementary strengths of CNNs and Transformers for improved image reconstruction.
• To enhance feature discriminability and achieve a better trade-off between model size and performance.

Main Methods:

• TC3Net employs a triple-branch structure integrating CNN Feature Extraction (CFE) blocks and Transformer Feature Extraction (TFE) blocks.
• Coupled Contrastive Blocks (CCBs), comprising Coupled Attention Blocks (CABs) and Local-Global Feature Extraction (LGFE) blocks, fuse feature maps and extract coupled information.
• A contrastive loss is introduced between CNN and Transformer feature maps to enhance discriminative characteristics.

Main Results:

• TC3Net demonstrates superior performance compared to several state-of-the-art (SOTA) SISR methods.
• The proposed network achieves a favorable balance between model size and reconstruction quality.
• Experimental results validate the effectiveness of the integrated CNN and Transformer approach.

Conclusions:

• TC3Net offers an effective unified framework for SISR by leveraging the strengths of both CNNs and Transformers.
• The CCB module plays a crucial role in feature fusion and information extraction for enhanced image reconstruction.
• The proposed method represents a significant advancement in achieving high-performance, computationally efficient SISR.