CrossDiff: Exploring Self-SupervisedRepresentation of Pansharpening via Cross-Predictive Diffusion Model.

Area of Science:

• Remote Sensing
• Computer Vision
• Artificial Intelligence

Background:

• Pansharpening merges spatial detail from panchromatic (PAN) images with spectral information from multispectral (MS) images.
• Deep learning models struggle with scale variations when trained on reduced resolutions for pansharpening.
• Existing methods often yield sub-optimal results due to the lack of high-resolution MS images during training.

Purpose of the Study:

• To develop a self-supervised representation learning method for pansharpening.
• To address the scale variation problem in deep learning-based pansharpening.
• To improve the fusion of spatial and spectral information from PAN and MS images.

Main Methods:

• Proposed a cross-predictive diffusion model named CrossDiff.
• Implemented a two-stage training strategy: pre-training UNet with a cross-predictive pretext task using conditional Denoising Diffusion Probabilistic Models (DDPM).
• Froze UNet encoders in the second stage to extract features, training only the fusion head for the pansharpening task.

Main Results:

• CrossDiff demonstrated superior performance compared to state-of-the-art supervised and unsupervised pansharpening methods.
• Extensive experiments validated the model's effectiveness and superiority.
• Cross-sensor experiments confirmed the generalization ability of the self-supervised representation learners across different satellite datasets.

Conclusions:

• The proposed CrossDiff model effectively enhances pansharpening by leveraging self-supervised learning.
• The two-stage training approach successfully mitigates scale variation issues.
• The model exhibits strong generalization capabilities for diverse remote sensing datasets.