Dual-Branch Multi-Perspective Modulation Network for Efficient Infrared Image Super-Resolution

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Transformer-based super-resolution methods excel by using self-attention for non-local information.
• However, self-attention in these methods incurs high computational costs and limits local detail learning due to its low-pass nature.
• Efficient infrared image super-resolution requires addressing these limitations.

Purpose of the Study:

• To propose an efficient Dual-Branch Multi-Perspective Modulation Network (DMMN) for infrared image super-resolution.
• To overcome the computational overhead and local detail learning restrictions of traditional transformer methods.
• To achieve a superior balance between reconstruction accuracy and computational efficiency.

Main Methods:

• Developed a Multi-Scale Feature Modulation Enhancement Unit (MFMEU) to capture cross-scale spatial features.
• Designed a Frequency-Domain Cross-Correlation Patch Modulation Unit (FCPMU) for global feature representation.
• Integrated an efficient Bidirectional Cross Modulation Unit (BCMU) to enhance feature interaction between MFMEU and FCPMU outputs.

Main Results:

• The DMMN demonstrated a competitive trade-off between reconstruction accuracy and computational efficiency.
• Achieved an average gain of 0.08 dB in PSNR compared to SRFormer-light across five public datasets.
• Operated 2.7x faster and utilized only 24% of the FLOPs compared to SRFormer-light.

Conclusions:

• The proposed DMMN effectively addresses the limitations of existing transformer-based super-resolution techniques.
• DMMN offers significant improvements in both speed and computational efficiency for infrared image super-resolution.
• The network architecture provides a promising solution for high-performance, efficient image reconstruction tasks.