Single-shot hyperspectral imaging based on dual attention neural network with multi-modal learning

Area of Science:

• Optics and Photonics
• Computer Vision
• Artificial Intelligence

Background:

• Hyperspectral imaging (HSI) is crucial for applications like medical diagnostics, sensing, and surveillance.
• Existing HSI techniques are often complex, requiring multiple alignment-sensitive components and pre-set parameters.
• There is a need for simpler, more efficient HSI methods.

Purpose of the Study:

• To develop an end-to-end snapshot hyperspectral imaging technique.
• To create a physics-informed dual attention neural network for hyperspectral data reconstruction.
• To enable direct hyperspectral volume recovery from a single RGB image.

Main Methods:

• A physics-informed dual attention neural network with multimodal learning was developed.
• The 3D spectral cube reconstruction was modeled as a compressive-imaging inverse problem.
• Spectra features and camera spectral sensitivity were jointly leveraged with an attention mechanism.

Main Results:

• The proposed method directly recovers hyperspectral volume from a single RGB image.
• Achieved ultra-fast performance compared to traditional scanning methods.
• Demonstrated 3.4 times higher precision than existing hyperspectral imaging convolutional neural networks.

Conclusions:

• The developed technique offers a simple, effective, and flexible approach to hyperspectral imaging.
• It eliminates the need for bulky setups and strict experimental limitations.
• Presents significant potential for diverse applications including pathological digital stain, computational imaging, and augmented reality.