Learning Time-multiplexed phase-coded apertures for snapshot spectral-depth imaging.

Area of Science:

• Computational imaging
• Optical engineering
• Computer vision

Background:

• Spectral-depth (SD) imaging is crucial for many applications but conventionally studied separately.
• Existing single snapshot SD imaging relies on optical modulation (codification) and computational decoding.
• Current optical modulation methods include coded apertures, phase masks, and active illumination.

Purpose of the Study:

• To propose a novel optical modulation strategy for joint spectral-depth imaging.
• To enhance the performance of single snapshot SD imaging systems.
• To develop an end-to-end framework integrating optical modulation and computational decoding.

Main Methods:

• Developed a time-multiplexed coded aperture (TMCA) using a color-coded aperture (CCA), time-varying phase-coded aperture, and pixel shutter.
• Utilized a spatially-variant point spread function (PSF) for improved depth distinguishability.
• Employed deep learning for an end-to-end (E2E) joint learning of optical modulation and computational decoding.
• Leveraged CCA for selective spectral band filtering to encode spectral information.

Main Results:

• The TMCA strategy enables better recovery of depth information due to its spatially-variant PSF.
• Spectral information is effectively encoded and disentangled using the CCA and reconstruction algorithm.
• Simulations and prototype experiments demonstrated superior performance compared to state-of-the-art SD imaging techniques.
• The E2E deep learning framework successfully integrated optical codification and computational decoding.

Conclusions:

• The proposed TMCA strategy offers a significant advancement in single snapshot spectral-depth imaging.
• Jointly learning optical modulation and computational decoding via deep learning improves reconstruction quality.
• The TMCA system demonstrates robust performance in both spectral and depth estimation.