An Asynchronous Linear Filter Architecture for Hybrid Event-Frame Cameras.

Area of Science:

• Computer Vision
• Robotics
• Sensor Fusion

Background:

• Event cameras excel at High Dynamic Range (HDR) and motion but struggle with static scenes.
• Conventional image sensors perform well in static scenes but are limited by HDR and motion.
• Existing methods often fail to leverage the complementary strengths of both sensor types.

Purpose of the Study:

• To develop a novel asynchronous linear filter architecture for fusing event and frame camera data.
• To enable robust HDR video reconstruction and spatial convolution for real-time robotic systems.
• To exploit the advantages of both event and frame-based sensing modalities.

Main Methods:

• An asynchronous linear filter architecture is proposed, integrating a state for encoding image information.
• The state is updated asynchronously by incoming data from event and frame cameras.
• The architecture supports real-time readout for subsequent vision modules and integrates spatial convolution kernels.

Main Results:

• The proposed AKF pipeline significantly outperforms state-of-the-art methods in HDR video reconstruction.
• Achieved a 69.4% reduction in absolute intensity error and a 35.5% improvement in image similarity indexes.
• Demonstrated successful integration of Gaussian, Sobel, and Laplacian spatial kernels.

Conclusions:

• The developed architecture effectively fuses event and frame camera data for superior HDR video reconstruction.
• The system provides a versatile and efficient solution for real-time robotic vision tasks.
• This fusion approach overcomes the limitations of individual sensor types, enhancing visual data processing.