Super-Resolving Dynamic Scenes With Spike Camera via Multi-Frame Sequential Alignment With Motion Propagation

Area of Science:

• Neuromorphic Engineering
• Computer Vision
• Image Processing

Background:

• Spike cameras offer high temporal resolution for dynamic scenes.
• Reconstructing high-resolution images from spike streams is challenging due to fluctuations and motion.

Purpose of the Study:

• To develop a super-resolution network for spike cameras.
• To address intensity extraction and temporal alignment issues in high-speed scenes.

Main Methods:

• A region-adaptive temporal filtering module to mitigate spike fluctuations and extract intensity.
• A multi-frame feature alignment module using long-term temporal information.
• Motion information propagation from neighboring moments to aid alignment.

Main Results:

• The proposed network effectively mitigates spike fluctuations.
• Accurate temporal alignment is achieved even with large motions.
• State-of-the-art performance demonstrated on synthetic and real spike data.

Conclusions:

• The developed spike camera super-resolution network enhances image reconstruction.
• The network successfully addresses key challenges in processing spike stream data.
• This work advances high-speed dynamic scene imaging using neuromorphic sensors.