E2VIDX: improved bridge between conventional vision and bionic vision.

Area of Science:

• Computer Vision
• Machine Learning
• Biomimetic Engineering

Background:

• Traditional cameras (RGBD, CMOS, CCD) struggle with motion blur and exposure in challenging lighting.
• Event cameras, inspired by biological vision, offer low latency, high dynamic range, and no motion blur.
• Event camera data requires specialized image reconstruction for compatibility with standard vision algorithms.

Purpose of the Study:

• To develop an improved image reconstruction network for event cameras.
• To enhance feature fusion and reduce model size compared to existing methods.
• To create a more versatile event camera reconstruction technique for practical applications.

Main Methods:

• Designed E2VIDX, a new neural network based on the E2VID method.
• Incorporated group convolution and sub-pixel convolution for enhanced feature fusion and model compression (25% size reduction).
• Developed a novel two-part loss function targeting both high-level and low-level features of reconstructed images.

Main Results:

• E2VIDX significantly outperforms state-of-the-art methods in image reconstruction quality.
• Achieved a 1.3% increase in Structural Similarity (SSIM) and a 1.7% decrease in Learned Perceptual Image Patch Similarity (LPIPS).
• Demonstrated faster processing speeds on both GPU and CPU, with a 2.5% reduction in Mean Squared Error (MSE).

Conclusions:

• E2VIDX provides superior event camera image reconstruction.
• The network's efficiency and performance allow direct application of existing vision algorithms.
• Reconstructed images from E2VIDX facilitate successful use in image classification, object detection, and instance segmentation tasks.