Multi-level spatial details cross-extraction and injection network for hyperspectral pansharpening

Area of Science:

• Remote Sensing
• Computer Vision
• Image Processing

Background:

• Hyperspectral (HS) imaging provides rich spectral information but often suffers from low spatial resolution.
• Pansharpening fuses HS images with high spatial resolution panchromatic (PAN) images to enhance spatial detail.
• Existing deep learning methods, particularly Convolutional Neural Networks (CNNs), often lack interpretability due to their black-box nature.

Purpose of the Study:

• To develop an interpretable deep learning framework for HS pansharpening.
• To address the interpretability limitations of current CNN-based pansharpening techniques.
• To improve the quality of fused HS images by effectively leveraging multi-resolution spatial information.

Main Methods:

• Proposes a Multi-level Spatial Details Cross-extraction and Injection Network (MSCIN).
• Integrates Multi-resolution Analysis (MRA) principles into a neural network architecture.
• Employs two specifically designed, interpretable sub-networks for details extraction and injection.

Main Results:

• The proposed MSCIN network demonstrates superior performance in HS pansharpening.
• Experimental results on widely used datasets validate the effectiveness of the method.
• The interpretable sub-networks successfully estimate missing details and injection gains.

Conclusions:

• The MSCIN offers an interpretable and effective solution for hyperspectral pansharpening.
• Integrating MRA concepts into deep learning enhances the understanding and performance of pansharpening models.
• The method provides a valuable advancement for HS image fusion applications.