Physics-Informed Masked Autoencoder for active sparse imaging.

Area of Science:

• Photonics and Artificial Intelligence
• Computer Vision and Machine Learning
• Applied Physics and Engineering

Background:

• Single-photon imaging technologies are advancing rapidly for applications like autonomous driving and biomedical imaging.
• Conventional methods struggle with long acquisition times and require dense, low-noise data for image formation.
• Sparse and noisy photon-counting data presents a significant challenge in current single-photon imaging systems.

Purpose of the Study:

• To introduce a novel approach, Physics-Informed Masked Autoencoder (PI-MAE), for fast and efficient data acquisition and image reconstruction in single-photon imaging.
• To demonstrate the effectiveness of PI-MAE in hardware-implemented Masked Autoencoder systems.
• To evaluate PI-MAE's performance in photon-limited scenarios, specifically within a single-photon LiDAR system.

Main Methods:

• Implementation of a hardware-accelerated Masked Autoencoder (MAE) architecture, termed Physics-Informed Masked Autoencoder (PI-MAE).
• Training and testing the PI-MAE model on digitally masked MNIST data within a single-photon LiDAR system simulation.
• Evaluating image reconstruction quality under conditions of extremely low photon counts (e.g., $$ photons per pulse, down to 9 photons per pixel) and high physical masking (90%).

Main Results:

• PI-MAE achieved high-quality image reconstruction on unseen object classes despite significant data sparsity and noise.
• Effective performance was demonstrated with minimal detected photons per pulse and as few as 9 detected photons per pixel.
• The system successfully reconstructed images with up to 90% physical masking, showcasing robustness.

Conclusions:

• PI-MAE offers a significant improvement for single-photon imaging systems, particularly in photon-starving applications.
• The developed approach acts as a viable hardware accelerator, enhancing data acquisition and image reconstruction efficiency.
• This work paves the way for improved performance in diverse fields relying on single-photon detection.