Snapshot video through dynamic scattering medium based on deep learning.

Area of Science:

• Optics and Photonics
• Computer Vision
• Machine Learning

Background:

• Dynamic object imaging through scattering media is challenging due to light distortion.
• Traditional methods struggle with high compression ratios and complex scattering environments.
• Compressive sensing offers potential but requires sophisticated reconstruction algorithms.

Purpose of the Study:

• To develop an end-to-end deep learning model for reconstructing multiple frames from a single snapshot.
• To enable high-speed imaging of dynamic objects in unknown, time-varying scattering conditions.
• To achieve significant data compression (up to 8X) while maintaining high image fidelity.

Main Methods:

• Integration of a coded aperture compressive temporal imaging system.
• Development of a specialized transformer-based convolutional neural network (CNN) for demultiplexing and reconstruction.
• Utilizing a dual-input CNN architecture incorporating speckle patterns and their autocorrelations.

Main Results:

• Successful reconstruction of up to eight frames from a single snapshot.
• Demonstrated compression ratio of up to 8X in both simulations and experiments.
• Achieved high reconstruction quality and accuracy, validated by ablation studies.
• The dual-input CNN significantly enhanced reconstruction accuracy compared to single-input models.

Conclusions:

• The proposed deep learning model effectively reconstructs dynamic scenes from compressed single-snapshot data.
• The integration of coded aperture imaging and advanced CNNs offers a powerful solution for high-speed imaging through scattering media.
• The dual-input CNN approach is crucial for maximizing reconstruction accuracy in complex scattering environments.