Spatially Varying Prior Learning for Blind Hyperspectral Image Fusion

Area of Science:

• Remote Sensing
• Computer Vision
• Image Processing

Background:

• Hyperspectral image fusion (HIF) aims to create high-resolution hyperspectral images (HR-HSI) from low-resolution hyperspectral (LR-HSI) and high-spatial-resolution multispectral (HR-MSI) data.
• Traditional and deep learning methods often rely on known degradation or paired training data, which are frequently unavailable in real-world scenarios.
• Existing HIF approaches may not fully leverage physical models or adequately address spatially-varying degradation.

Purpose of the Study:

• To propose a novel deep blind HIF method that overcomes limitations of existing approaches.
• To develop a method that does not require paired HR-LR training data and can handle unknown, spatially-varying degradation.
• To improve the accuracy and visual quality of reconstructed HR-HSI.

Main Methods:

• Unfolding a model-based maximum a posteriori (MAP) estimation into a deep network for blind HIF.
• Utilizing a Laplace distribution (LD) prior, eliminating the need for paired training data.
• Developing an observation module to learn spatial degradation directly from LR-HSI data.
• Employing a Swin-Transformer-based denoiser to learn uncertainty in LD models and estimating degraded image variance from residual errors.
• Jointly optimizing MAP estimation parameters and the observation module via end-to-end training.

Main Results:

• The proposed deep blind HIF method demonstrates superior performance compared to existing methods.
• Objective evaluation metrics show significant improvements in reconstructed HR-HSI quality.
• Visual assessment confirms enhanced detail and fidelity in the fused images on both synthetic and real datasets.

Conclusions:

• The developed deep blind HIF method effectively reconstructs HR-HSI without requiring paired training data or prior knowledge of degradation.
• The approach successfully addresses the challenge of spatially-varying degradation through a learned observation module.
• The method offers a robust and efficient solution for hyperspectral image fusion, outperforming current state-of-the-art techniques.