Neuromorphic Imaging With Joint Image Deblurring and Event Denoising

Pei Zhang , Haosen Liu , Zhou Ge

IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society +

|

March 12, 2024

View abstract on PubMed

Summary

This study introduces a novel algorithm that combines neuromorphic camera events and blurry images to reconstruct clear images and clean event data. This approach enhances both data types for improved machine processing and reasoning.

Area Of Science

Computer Vision
Neuromorphic Engineering
Image Processing

Background

Neuromorphic imaging offers high temporal precision but suffers from noisy events and blurry images of fast-moving objects.
These data deficiencies hinder effective human observation and machine processing.
Events capture object edges and motion details, complementing latent sharp images.

Purpose Of The Study

To develop a unifying algorithm for joint reconstruction of blur-free images and noise-robust events from neuromorphic imaging data.
To leverage the complementary nature of event streams and intensity images for improved data quality.
To enhance the robustness and efficiency of neuromorphic data processing.

Main Methods

An iterative coarse-to-fine algorithm was developed to fuse event and image data.
Event-regularized priors were used for blind deblurring, incorporating high-frequency structures and dynamic features.
Image gradients provided supervision for neuromorphic noise removal.

Main Results

The synergistic approach significantly improved reconstruction quality for both motion-blurred images and noisy event streams.
The method demonstrated superior robustness across varying illumination, contrast, and motion magnitudes.
The algorithm requires fewer events and offers competitive computational efficiency.

Conclusions

The proposed method effectively addresses the limitations of raw neuromorphic data, producing high-quality images and clean event streams.
This technique enhances the utility of neuromorphic sensing for tasks requiring accurate motion analysis and reasoning.
The solution is suitable for resource-limited computing environments, promoting wider adoption of neuromorphic imaging.