$S^{2}$S2-Transformer for Mask-Aware Hyperspectral Image Reconstruction.

Area of Science:

• Optics and Photonics
• Computer Vision
• Machine Learning

Background:

• Snapshot compressive imaging (SCI) captures hyperspectral images using optical compression and software reconstruction.
• Coded aperture snapshot compressive imager (CASSI) with Transformer reconstruction shows promise but faces limitations in hyperspectral modeling due to attention designs and information entanglement.
• CASSI's physical mask causes data loss, hindering reconstruction fidelity.

Purpose of the Study:

• To propose a novel spatial-spectral ($S^{2}$S2-) Transformer to overcome limitations in CASSI hyperspectral image reconstruction.
• To enhance the modeling of entangled spatial and spectral information and mitigate data loss from physical masks.
• To advance physics-driven deep network design for improved CASSI reconstruction.

Main Methods:

• Implemented a spatial-spectral ($S^{2}$S2-) Transformer with a parallel attention design for disentangled spatial and spectral information modeling.
• Introduced a mask-aware learning strategy to adaptively prioritize loss penalties for masked regions, using mask-encoded predictions as uncertainty estimators.
• Theoretically analyzed convergence tendencies for masked and unmasked regions under the proposed learning strategy.

Main Results:

• The parallel attention design effectively disentangles and models blended spatial and spectral information.
• The mask-aware learning strategy improves reconstruction accuracy by addressing masked data loss.
• Extensive experiments show the proposed method outperforms state-of-the-art methods in hyperspectral image reconstruction.

Conclusions:

• The proposed spatial-spectral ($S^{2}$S2-) Transformer with mask-aware learning significantly enhances hyperspectral image reconstruction quality in CASSI systems.
• Parallel attention and adaptive loss weighting are effective strategies for improving deep network design in SCI.
• The method offers a more robust and accurate approach to hyperspectral imaging compared to existing techniques.