Multistage Spatial-Spectral Fusion Network for Spectral Super-Resolution.

Area of Science:

• Computer Vision
• Image Processing
• Deep Learning

Background:

• Deep learning methods show promise for spectral super-resolution (SSR).
• Existing SSR methods often fail to adequately model both spatial and spectral features in hyperspectral images (HSI).
• This limitation leads to a loss of discriminative information and reduced fidelity in reconstructed HSIs.

Purpose of the Study:

• To propose a novel SSR network, the multistage spatial-spectral fusion network (MSFN).
• To improve the modeling of spatial-spectral features for enhanced HSI reconstruction.
• To overcome the limitations of existing methods in capturing comprehensive spatial correlations and spectral self-similarity.

Main Methods:

• Developed a multistage Unet-like architecture for capturing multiscale spatial and spectral features.
• Incorporated two types of self-attention mechanisms for comprehensive global modeling of HSI.
• Introduced innovative spatial fusion (SpatialFM) and spectral fusion (SpectralFM) modules for feature alignment and fusion.

Main Results:

• The proposed MSFN effectively captures and fuses multiscale spatial-spectral features.
• Experiments on NTIRE2022 and NTIRE2020 datasets show superior performance compared to state-of-the-art SSR methods.
• Quantitative and qualitative evaluations confirm enhanced accuracy in reconstructed HSIs.

Conclusions:

• The MSFN significantly enhances the fidelity of hyperspectral image reconstruction.
• The network's ability to comprehensively model spatial-spectral features is key to its improved performance.
• The proposed fusion modules effectively preserve and integrate spatial and spectral information.