Ultra-high-speed four-dimensional hyperspectral imaging.

Area of Science:

• Optics and Photonics
• Computer Vision
• Deep Learning

Background:

• Traditional spectral imaging systems often face limitations in speed and spatial resolution.
• Acquiring simultaneous spectral and depth information in real-time remains a significant challenge in imaging science.

Purpose of the Study:

• To introduce a novel deep learning-enabled four-dimensional spectral imaging system.
• To achieve simultaneous acquisition of high-resolution spectral data cubes and depth information.
• To enhance the spatial reconstruction of spectral images through data fusion.

Main Methods:

• A reflective coded aperture snapshot spectral imaging system integrated with a panchromatic camera.
• Utilizing a U-net-3D network for hyperspectral data cube recovery.
• Employing stereo matching to estimate disparity maps for depth information extraction.
• Implementing a designed fusion network to integrate panchromatic measurements for improved spatial resolution.

Main Results:

• The system successfully captures compressively coded hyperspectral and panchromatic measurements simultaneously.
• Depth information is accurately retrieved, enabling scene depth mapping.
• The fusion network enhances the spatial reconstruction of the hyperspectral data cube.
• The hardware prototype demonstrated high-speed 4D spectral imaging with 8 nm spectral resolution (450-700 nm), 2.5 mm depth accuracy, and 1.83 s reconstruction time.

Conclusions:

• The proposed system offers a novel and effective approach for high-speed four-dimensional spectral imaging.
• Simultaneous acquisition of spectral and depth information is achieved with high accuracy and resolution.
• Deep learning, particularly U-net-3D and fusion networks, plays a crucial role in enabling this advanced imaging capability.