Event-Aware Video Deraining via Multi-Patch Progressive Learning.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Neuromorphic Engineering

Background:

• Video-based rain streak removal is challenging due to complex spatio-temporal correlations.
• Existing deraining methods struggle to effectively model these characteristics.
• Neuromorphic cameras offer unique event-based data capturing intensity changes.

Purpose of the Study:

• To develop an advanced neural network for effective video rain streak removal.
• To integrate event data from neuromorphic cameras to improve deraining.
• To propose a novel multi-patch progressive network architecture.

Main Methods:

• An event-aware module was developed to encode data from neuromorphic cameras.
• A multi-patch progressive neural network was designed for deraining.
• The network utilizes varying receptive fields and progressive learning across patch levels.

Main Results:

• The proposed method demonstrated superior performance in video rain streak removal.
• Event data integration significantly facilitated the deraining process.
• Outperformed state-of-the-art methods on both synthetic and real-world datasets.

Conclusions:

• Event-aware processing is a valuable approach for enhancing video deraining.
• The developed multi-patch progressive neural network effectively removes rain streaks.
• The method shows strong potential for real-world applications requiring clear video footage.