Semantic information guided diffusion posterior sampling for remote sensing image fusion.

Area of Science:

• Remote Sensing
• Computer Vision
• Artificial Intelligence

Background:

• Image fusion combines optical and SAR data, but diffusion models struggle with SAR speckle and lack high-level semantic information.
• Existing methods integrate pixel-level features, limiting accuracy in tasks like target detection and classification.

Purpose of the Study:

• To propose a novel semantic information guided diffusion posterior sampling method for improved optical and SAR image fusion.
• To address limitations of current diffusion models in handling SAR speckle and incorporating semantic context for enhanced downstream applications.

Main Methods:

• Preprocessing using SAR-BM3D to reduce speckle in SAR images.
• Developing a diffusion posterior sampling model incorporating fidelity, regularization, and semantic guidance terms.
• Utilizing variational diffusion with variational inference and stochastic optimization for fidelity and regularization, and a designed FLCNet for semantic guidance via cross-entropy loss.

Main Results:

• The proposed method effectively integrates information from optical and SAR images, overcoming SAR speckle interference.
• The semantic guidance significantly improves the quality of fused images for high-level tasks.
• Experimental validation on WHU-OPT-SAR and DDHRNet datasets demonstrates the method's feasibility and superiority.

Conclusions:

• Semantic information guided diffusion posterior sampling offers a robust solution for optical and SAR image fusion.
• The method enhances fused image utility for critical applications such as target detection and classification.
• This approach represents a significant advancement in leveraging diffusion models for multi-modal remote sensing image fusion.