MHF-Net: An Interpretable Deep Network for Multispectral and Hyperspectral Image Fusion

Area of Science:

• Remote Sensing
• Computer Vision
• Image Processing

Background:

• Multispectral and hyperspectral (MS/HS) image fusion is crucial for generating high-resolution hyperspectral (HrHS) images.
• Existing methods often lack interpretability and struggle with data from different sensors.

Purpose of the Study:

• To develop an interpretable and generalizable deep learning network for MS/HS image fusion.
• To address challenges in fusing images with inconsistent spectral and spatial responses.

Main Methods:

• Designed MHF-net, a network incorporating linear mapping and low-rank priors.
• Unfolded proximal gradient algorithms to build the network architecture.
• Developed two regimes: consistent MHF-net and blind MHF-net for varying data conditions.

Main Results:

• MHF-net modules possess clear physical meanings, enhancing interpretability.
• Achieved superior visual and quantitative results on simulated and real datasets.
• Blind MHF-net demonstrated strong generalization across different sensors and response mismatches.

Conclusions:

• MHF-net offers an interpretable and effective solution for MS/HS image fusion.
• The blind MHF-net regime significantly improves generalization capabilities in challenging scenarios.
• The proposed method outperforms state-of-the-art techniques in both visual and quantitative evaluations.