DSMT: Dual-Stage Multiscale Transformer for Hyperspectral Snapshot Compressive Imaging

Area of Science:

• Image Processing
• Computer Vision
• Spectroscopy

Background:

• Snapshot compressive imaging (SCI) enables efficient 3D hyperspectral image (HSI) acquisition by compressing data into 2D measurements.
• Reconstructing high-quality HSIs from SCI data is challenging due to the inverse problem's complexity.
• Existing Transformer-based methods struggle with capturing multi-scale features and local information efficiently.

Purpose of the Study:

• To propose a novel Dual-Stage Multiscale Transformer (DSMT) for improved HSI reconstruction from compressed measurements.
• To enhance reconstruction accuracy, network generalization, and computational efficiency in HSI reconstruction.
• To address the limitations of current Transformer methods in capturing multi-scale and local features.

Main Methods:

• Developed a U-Net architecture with a dual-branch encoder for processing distinct features and refined reconstruction.
• Incorporated full-scale skip connections to strengthen feature fusion across different network stages.
• Introduced a dual-window multiscale multi-head self-attention (DWM-MSA) mechanism for capturing multi-scale dependencies and local information.
• Implemented a hybrid positional embedding (CRPE) to dynamically model spatial and spectral dependencies.

Main Results:

• The proposed DSMT method demonstrated superior performance in quantitative and qualitative experiments on simulated and real HSI data.
• The dual-stage, multiscale approach significantly improved HSI reconstruction quality.
• The DWM-MSA and CRPE components effectively enhanced the model's ability to capture complex spatial-spectral information.

Conclusions:

• The DSMT framework offers a robust and effective solution for HSI reconstruction from SCI measurements.
• The method achieves high reconstruction accuracy, stability, and generalization capabilities.
• The proposed architecture and attention mechanisms represent a significant advancement in hyperspectral imaging reconstruction.