Optical linear systems framework for event sensing and computational neuromorphic imaging

Area of Science:

• Computational Imaging
• Sensor Technology
• Optics

Background:

• Event vision sensors (neuromorphic cameras) provide sparse, asynchronous data with high dynamic range and microsecond-scale sensing.
• The event-based data representation challenges conventional physics-based sensor design and integration with standard computational imaging forward operators.

Purpose of the Study:

• To develop a physics-grounded processing pipeline for event vision sensor data.
• To enable direct scene inference from event streams, even without a traditional image formation.
• To bridge the gap between event sensing and model-based computational imaging for dynamic optical systems.

Main Methods:

• Mapping event streams to per-pixel log-intensity and its derivatives.
• Embedding event measurements into a dynamic linear systems model with a time-varying Point-Spread Function (PSF).
• Implementing inverse filtering via frequency-domain Wiener deconvolution using a known dynamic transfer function.

Main Results:

• Validated the approach in simulations and real-world data from a tunable-focus telescope imaging a star field.
• Demonstrated superior source localization and separability compared to traditional frame-based imaging.
• Enabled scene inference directly from event data, bypassing the need for a formed image.

Conclusions:

• The proposed physics-grounded pipeline effectively processes event data for computational imaging applications.
• This framework facilitates model-based inverse filtering and scene inference from dynamic optical systems using event vision sensors.
• The approach offers a practical solution for integrating neuromorphic sensing with advanced imaging techniques.