PHoM: Effective pan-sharpening via higher-order state-space model

Area of Science:

• Remote Sensing
• Computer Vision
• Signal Processing

Background:

• Pan-sharpening generates high-resolution multi-spectral images from low-resolution multi-spectral and high-resolution panchromatic data.
• Mamba-based models excel at long-range relational modeling but struggle with higher-order spectral feature interactions.

Purpose of the Study:

• To propose a novel higher-order state-space model (PHoM) for pan-sharpening.
• To enhance the modeling of spectral feature interactions beyond first-order mappings.
• To improve the representation capability for multi-spectral and panchromatic image fusion.

Main Methods:

• Introduced a higher-order state-space model (PHoM) based on splitting, interaction, and aggregation.
• Developed a cross-modal PHoM to capture higher-order cross-modal correlations.
• Conducted extensive experiments on diverse datasets to validate performance.

Main Results:

• The proposed PHoM effectively models higher-order spatial adaptive interactions.
• Cross-modal PHoM significantly improves representation by exploiting cross-modal correlations.
• Experimental results demonstrate substantial performance gains over state-of-the-art methods.

Conclusions:

• PHoM offers a superior approach to pan-sharpening by addressing limitations in modeling spectral feature interactions.
• The cross-modal extension further enhances fusion capabilities, leading to state-of-the-art results.
• This work advances high-resolution multi-spectral image generation techniques.