Multi-Stage Network for Event-Based Video Deblurring with Residual Hint Attention

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Traditional video deblurring methods struggle with severely blurred frames.
• Event cameras offer advantages like low latency and reduced motion blur for deblurring.
• Direct use of event data can introduce artifacts like noise and incorrect textures.

Purpose of the Study:

• To develop an improved video deblurring method using event camera data.
• To overcome the limitations of existing frame-based and direct event-based deblurring techniques.
• To enhance the quality of deblurred video frames, especially in challenging scenarios.

Main Methods:

• Proposed a two-stage coarse-refinement network for event-based video deblurring.
• The first stage performs event-based deblurring to estimate a coarse frame.
• The second stage refines the coarse frame using available video frames and a proposed Residual Hint Attention (RHA) module.

Main Results:

• Achieved superior deblurring performance compared to frame-based methods, even with severely blurred inputs.
• The proposed network demonstrated significant improvements on the GoPro and HQF datasets.
• Outperformed the state-of-the-art D2Net method by 1 dB PSNR and 0.05 SSIM on GoPro, and 1.7 dB PSNR and 0.03 SSIM on HQF.

Conclusions:

• The two-stage coarse-refinement network effectively addresses artifacts in event-based deblurring.
• Combining event data with frame-based refinement yields higher quality deblurred videos.
• The RHA module plays a crucial role in guiding the refinement process for superior results.