EDVD: Cross-Modal Spatio-Temporal Fusion With Event and Diffusion for Video Deblurring

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Video deblurring from severely blurred scenes remains a significant challenge.
• Event-based methods show promise but struggle with feature fusion due to modal differences and event sparsity.
• Existing approaches face difficulties in restoring fine local details and textures in deblurred videos.

Purpose of the Study:

• To develop an advanced video deblurring method that overcomes limitations of current event-based techniques.
• To enhance the fusion of information from blurred image frames and event data.
• To improve the restoration of local details and textures in deblurred videos.

Main Methods:

• A cross-modal collaborative attention mechanism is proposed for effective feature fusion between blurred and event frames.
• A diffusion model is employed to generate spatial guiding prior features for detail enhancement.
• An event-guided dynamic feature fusion module adaptively integrates spatio-temporal information.

Main Results:

• The proposed method demonstrates superior performance in restoring high-quality images from blurred videos.
• Experimental results on synthetic and real datasets show significant improvements over state-of-the-art methods.
• Enhanced extraction of motion information and improved restoration of local details and textures were observed.

Conclusions:

• The developed method effectively addresses the challenges of modal differences and event sparsity in video deblurring.
• The integration of cross-modal attention, diffusion models, and dynamic feature fusion yields state-of-the-art results.
• This approach offers a promising solution for high-quality image restoration in severely blurred video sequences.