Toward Resolution Mismatching: Modality-Aware Feature-Aligned Network for Pan-Sharpening.

Area of Science:

• Remote Sensing
• Image Processing
• Computer Vision

Background:

• Panchromatic (PAN) and multi-spectral (MS) image fusion (pan-sharpening) aims to enhance spatial resolution of MS images using PAN data.
• Current methods struggle with spatial resolution mismatch between PAN and MS images, causing feature misalignment and artifacts.
• This misalignment hinders high-frequency texture generation and overall performance in pan-sharpening.

Purpose of the Study:

• To propose a novel modality-aware feature-aligned pan-sharpening framework.
• To address the spatial resolution mismatching problem inherent in current pan-sharpening techniques.
• To improve the quality of fused high-resolution multi-spectral satellite images.

Main Methods:

• A framework with three stages: modality-aware feature extraction, alignment, and context-integrated reconstruction.
• Utilizes half-instance normalization for consistent feature learning between PAN and MS modalities.
• Employs learnable modality-aware feature interpolation with predicted transformation offsets for adaptive feature alignment.

Main Results:

• The proposed framework effectively aligns features from PAN and MS images, mitigating misalignment issues.
• Demonstrates superior performance over state-of-the-art methods in qualitative and quantitative evaluations.
• Achieves enhanced generation of high-frequency textures and reduced blurry artifacts in fused images.

Conclusions:

• The novel framework successfully addresses the spatial resolution mismatch in pan-sharpening.
• The method shows significant improvements in image fusion quality and generalization ability.
• Offers a more effective approach for producing high-resolution multi-spectral satellite imagery.