Hyperspectral Image Super Resolution With Real Unaligned RGB Guidance

Area of Science:

• Computer Vision
• Image Processing
• Remote Sensing

Background:

• Hyperspectral image (HSI) super-resolution aims to enhance image detail by integrating spatial information from high-resolution (HR) RGB images.
• Existing fusion-based methods struggle with real-world scenarios due to rigid alignment assumptions and inability to handle nonrigid misalignments between HSI and RGB data.

Purpose of the Study:

• To develop a robust fusion-based HSI super-resolution method capable of handling both rigid and nonrigid misalignments in real-world reference RGB (Ref-RGB) images.
• To introduce a novel HSI fusion network (HSIFN) designed for unaligned HSI-RGB data.

Main Methods:

• Proposed HSI fusion network (HSIFN) employing heterogeneous feature extraction via separate HSI and RGB encoders.
• Implemented a multistage feature alignment module to explicitly align Ref-RGB features with low-resolution (LR) HSI features.
• Utilized an adaptive attention module to focus on discriminative regions before feature fusion and HR HSI reconstruction.

Main Results:

• The HSIFN demonstrated superior performance over existing single-image and fusion-based super-resolution methods on both simulated and real-world datasets.
• Achieved significant improvements in quantitative assessments and visual comparisons, validating its effectiveness for real scenes.
• Introduced a new real-world HSI fusion dataset to facilitate research on unaligned HSI-RGB super-resolution.

Conclusions:

• The proposed HSIFN effectively addresses the challenge of unaligned HSI-RGB data in fusion-based super-resolution.
• The method offers a significant advancement for HSI super-resolution in practical, real-world applications.
• Publicly released code and dataset encourage further research and development in this area.