Non-Uniform Exposure Imaging via Neuromorphic Shutter Control

Area of Science:

• Computer Vision
• Neuromorphic Engineering
• Image Processing

Background:

• Conventional cameras struggle with real-time motion perception for adaptive shutter control, limiting imaging in harsh environments.
• Non-uniform exposures offer flexibility but introduce challenges in managing motion blur and noise.
• Existing methods lack the ability to perceive intra-frame dynamic information for adaptive exposure control.

Purpose of the Study:

• To develop a novel Neuromorphic Shutter Control (NSC) system for real-time adaptive camera shutter control.
• To overcome limitations of conventional cameras in dynamic environments by leveraging event-based sensing.
• To enhance image quality by mitigating motion blur and noise through adaptive exposure and advanced denoising.

Main Methods:

• Proposed a Neuromorphic Shutter Control (NSC) system utilizing the low latency of event-based sensors for real-time motion monitoring and adaptive exposure.
• Developed an event-based image denoising network (SEID) using self-supervised learning to stabilize Signal-to-Noise Ratio (SNR) and denoise images.
• Explored image noise statistics and inter-frame motion information from events to generate self-supervision signals for SEID.
• Implemented the NSC system in a hybrid-camera prototype and collected a synchronized real-world dataset.

Main Results:

• The NSC system effectively avoids motion blur and alleviates instant noise in dynamic scenes.
• The SEID network successfully stabilizes inconsistent SNR caused by non-uniform exposure times.
• Experiments on synthetic and real-world datasets demonstrate superior performance compared to state-of-the-art methods.
• The hybrid-camera prototype validates the system's effectiveness in diverse real-world scenarios.

Conclusions:

• The proposed Neuromorphic Shutter Control (NSC) system offers a robust solution for adaptive imaging in challenging environments.
• Event-based sensing combined with self-supervised learning enables high-quality image acquisition with real-time motion adaptation.
• The developed system significantly advances the capabilities of adaptive camera shutter control for practical applications.