A novel pansharpening method based on cross stage partial network and transformer

Area of Science:

• Remote Sensing
• Computer Vision
• Image Processing

Background:

• Conventional Convolutional Neural Networks (CNNs) have limitations in capturing global features due to restricted receptive fields in remote sensing image fusion.
• Transformers offer global receptive fields via self-attention but incur high computational costs for high-resolution images.

Purpose of the Study:

• To develop a novel remote sensing image sharpening method that overcomes the limitations of existing CNNs and Transformers.
• To enhance the fusion performance by integrating the strengths of Guided Filtering (GF), Cross Stage Partial Network (CSPNet), and Transformer.

Main Methods:

• The proposed GF-CSTNet method utilizes Guided Filtering (GF) for initial image enhancement.
• It combines Cross Stage Partial Network (CSPNet) and Transformer architectures for improved fusion.
• A Rep-Conv2Former method with a multi-scale convolution modulator block and a reparameterization module is introduced for efficient feature extraction and optimized inference speed.

Main Results:

• Experimental results on GaoFen-2 and WorldView-3 datasets validate the effectiveness of GF-CSTNet.
• The method successfully extracts detailed image information.

Conclusions:

• GF-CSTNet provides an effective solution for remote sensing image sharpening.
• The approach achieves high-quality fusion results without spectral distortion, demonstrating its practical applicability.