Hyperspectral Image Super-Resolution via Knowledge-Driven Deep Unrolling and Transformer Embedded Convolutional...

Area of Science:

• Remote Sensing
• Computer Vision
• Image Processing

Background:

• Hyperspectral (HS) imaging offers rich spectral information but often suffers from low spatial resolution due to hardware constraints.
• Existing HS image fusion methods require a co-registered high-resolution auxiliary image, which is not always available in real-world scenarios.
• Low spatial resolution in HS images significantly degrades performance in various applications.

Purpose of the Study:

• To develop a novel single hyperspectral (HS) image super-resolution method.
• To address the limitation of requiring auxiliary images in traditional HS image fusion techniques.
• To enhance the spatial resolution of HS images using a knowledge-driven deep unrolling approach.

Main Methods:

• Proposes a maximum a posteriori (MAP) based energy model with implicit priors, solved via alternating optimization.
• Unrolls the iterative optimization mechanism using a Transformer-embedded convolutional recurrent neural network (CRNN).
• Integrates Vision Transformer and 3D convolution for spatial-spectral prior learning and recurrent connections for iterative reconstruction.

Main Results:

• Achieves effective knowledge-driven, end-to-end, and data-dependent HS image super-resolution.
• Demonstrates superior performance compared to state-of-the-art methods on three HS image datasets.
• Successfully enhances the spatial resolution of HS images without auxiliary data.

Conclusions:

• The proposed knowledge-driven deep unrolling technique offers a robust solution for single HS image super-resolution.
• The method overcomes the dependency on auxiliary images, expanding applicability in real-world scenarios.
• The integrated Transformer and CRNN architecture effectively captures spatial-spectral information for improved resolution.