Self-Expressive High-Order Tensor Unrolling Network for Unsupervised Hyperspectral and Multispectral Image Fusion

Area of Science:

• Remote Sensing
• Computer Vision
• Data Fusion

Background:

• Hyperspectral and multispectral image fusion (HMF) aims to enhance spatial-spectral quality by combining low-resolution hyperspectral images (LR-HSI) with high-resolution multispectral images (HR-MSI).
• Existing fusion methods face challenges, including disruption of spatial consistency in tensor-based approaches and lack of interpretability in deep learning methods.
• There is a need for unsupervised HMF methods that preserve spatial structures and offer interpretability.

Purpose of the Study:

• To propose a novel unsupervised hyperspectral and multispectral image fusion method named Self-Expressive High-Order Tensor Unrolling Network (SHOTUN).
• To address the limitations of existing fusion techniques, particularly regarding spatial consistency and interpretability.
• To improve the generalization capabilities of fusion models across different sensors.

Main Methods:

• Developed a Self-Expressive High-Order Tensor Unrolling Network (SHOTUN) within a sparse core tensor decomposition framework.
• Introduced self-expressive relationships among image patches for high-order mode representation to preserve spatial structure.
• Employed an alternative optimizing strategy with dedicated modules for an interpretable end-to-end training pipeline.
• Incorporated a pre-training strategy for unsupervised training to enhance the estimation of unknown degraded parameters.

Main Results:

• The proposed SHOTUN method effectively fuses hyperspectral and multispectral images, enhancing spatial-spectral quality.
• Experimental results on simulated and real datasets demonstrate the effectiveness of the SHOTUN method.
• The method preserves intrinsic spatial consistency and offers an interpretable fusion process.

Conclusions:

• SHOTUN provides an effective and interpretable solution for unsupervised hyperspectral and multispectral image fusion.
• The proposed method overcomes limitations of conventional tensor decomposition and deep learning fusion approaches.
• The pre-training strategy improves the model's generalization across diverse datasets and sensors.