Formulating Event-Based Image Reconstruction as a Linear Inverse Problem With Deep Regularization Using Optical Flow.

Area of Science:

• Computer Vision
• Robotics
• Sensor Technology

Background:

• Event cameras offer high dynamic range (HDR) and high-speed imaging by asynchronously measuring brightness changes.
• Reconstructing images from event data is crucial for applications in robotics and slow-motion HDR video generation.
• Current methods often rely on complex, difficult-to-tune Recurrent Neural Networks (RNNs) for event-to-image conversion.

Purpose of the Study:

• To develop a novel, explainable, and tunable method for event-based image reconstruction.
• To demonstrate that combining motion and brightness estimation allows formulation as a linear inverse problem.
• To solve event-based image reconstruction without training an image reconstruction RNN.

Main Methods:

• Formulating event-based image reconstruction as a linear inverse problem, integrating motion and brightness estimation.
• Employing classical and learning-based regularizers, including a denoising Convolutional Neural Network (CNN), to solve the inverse problem and refine images.
• Utilizing data from a short time interval for reconstruction.

Main Results:

• Achieved visual image quality comparable to state-of-the-art methods.
• Demonstrated state-of-the-art results using a CNN as the regularization function.
• Showcased the method's ability to reconstruct images with high dynamic range and high-speed properties.

Conclusions:

• The proposed linear inverse problem formulation offers an explainable and tunable alternative to RNN-based methods for event-based image reconstruction.
• The approach is versatile, applicable to reconstructing brightness from second derivatives and combinable with super-resolution, motion segmentation, and color demosaicing.
• This method advances robotic vision and HDR video generation by providing a more accessible and effective image reconstruction technique.