High Frame Rate Video Reconstruction Based on an Event Camera.

Area of Science:

• Computer Vision
• Robotics
• Computational Imaging

Background:

• Event-based cameras offer microsecond accuracy for high-speed motion and challenging lighting.
• Dynamic and Active-pixel Vision Sensors (DAVIS) capture both intensity frames and events simultaneously.
• Standard DAVIS outputs suffer from low frame rates and motion blur, limiting video quality.

Purpose of the Study:

• To develop an effective approach for reconstructing high-quality, high frame rate sharp videos from blurred images captured by event-based cameras.
• To model the blur-generation process by associating event data with latent sharp images.
• To improve temporal resolution and remove motion blur in video reconstruction.

Main Methods:

• Proposed the Event-based Double Integral (EDI) model, associating event data with latent sharp images to model blur.
• Extended the model to the multiple Event-based Double Integral (mEDI) for smoother results using multiple images and events.
• Developed a new, efficient solver to minimize the proposed energy model via non-convex optimization.

Main Results:

• Achieved significant improvements in removing motion blur and reconstructing high temporal resolution videos.
• Demonstrated superior performance of the mEDI model and optimization method on synthetic and real datasets.
• Successfully reconstructed sharp videos with high frame rates from low frame rate, blurred inputs.

Conclusions:

• The proposed mEDI model and its efficient solver offer a superior solution for high-quality video reconstruction from event-based camera data.
• This method effectively addresses the limitations of motion blur and low temporal resolution in current event-based vision systems.
• The approach holds promise for applications requiring high-speed, high-fidelity visual data acquisition.