EV-LFV: Synthesizing Light Field Event Streams from an Event Camera and Multiple RGB Cameras.

Area of Science:

• Computer Vision
• Immersive Media Technology
• Event-Based Sensing

Background:

• Light field videos (LFV) offer immersive 6-DoF experiences but suffer from low frame rates and motion blur due to resource-intensive processing.
• Existing LFV systems struggle with capturing fast motion effectively, limiting real-time applications.
• Event cameras offer high temporal resolution but are costly, hindering their use in multi-camera LFV setups.

Purpose of the Study:

• To develop an efficient framework (EV-LFV) for synthesizing multi-view event-based LFV using a single event camera and multiple RGB cameras.
• To overcome the limitations of traditional RGB-LFV systems in capturing fast motion and reducing motion blur.
• To create a comprehensive dataset for training and evaluating event synthesis models for LFV.

Main Methods:

• Proposed EV-LFV framework utilizing spatial-angular convolution, ConvLSTM, and Transformer architectures.
• Leveraged a single event camera alongside multiple RGB cameras to synthesize dense multi-view event streams.
• Constructed the first event-to-LFV dataset with 200 RGB-LFV sequences and ground-truth event streams for training.

Main Results:

• EV-LFV successfully synthesized full multi-subview event-based RGB-LFV.
• The proposed method demonstrated superior performance compared to state-of-the-art event synthesis techniques.
• Effectively reduced motion blur in reconstructed RGB-LFV, improving visual quality for fast-moving scenes.

Conclusions:

• EV-LFV provides a cost-effective solution for high-temporal-resolution LFV capture.
• The framework significantly enhances the quality of LFV, particularly for dynamic scenes.
• This work paves the way for more accessible and higher-fidelity immersive video experiences.