Learning based compressive snapshot spectral light field imaging with RGB sensors

Area of Science:

• Optics and Photonics
• Computer Vision
• Computational Imaging

Background:

• Multidimensional optical sensing, including spectral light field (SLF) imaging, is crucial for fields like computer vision and microscopy.
• Traditional SLF sensors often require expensive components such as beam-splitters or multiple cameras.
• There is a need for cost-effective SLF imaging solutions.

Purpose of the Study:

• To develop a low-cost method for capturing four-dimensional (4D) spectral light field data.
• To reconstruct SLF information using readily available RGB light field cameras.
• To provide an accessible alternative for advanced optical sensing.

Main Methods:

• A compressed snapshot imaging method based on RGB light field cameras.
• Utilizing a U-shaped neural network incorporating multi-head self-attention and unparameterized Fourier transform modules.
• Leveraging the principles of compressive sensing for data reconstruction.

Main Results:

• Successful reconstruction of 4D SLF data from RGB light field camera measurements.
• Achieved spectral resolution of 10 nm within the 400-700 nm range.
• Obtained angular resolution of 9x9 and spatial resolution of 622x432.

Conclusions:

• The proposed method offers a low-cost approach to SLF imaging.
• This technique enables the acquisition of rich spatial-spectral-angular information.
• It presents a viable alternative to conventional, high-cost SLF imaging systems.