A Multi-Stage Dual Encoder-Decoder Network Based on Event Image Cross-Modal Fusion for Image Deblurring

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Existing event-driven image deblurring methods often fail to account for modality differences, leading to feature mismatches.
• Lack of explicit alignment strategies in current methods degrades reconstruction quality.

Purpose of the Study:

• To propose a multi-stage dual encoder-decoder network for event-driven image deblurring.
• To enhance image reconstruction by effectively fusing event and frame image features.

Main Methods:

• Utilized dilated convolutional residual modules for feature extraction in the encoder.
• Integrated a cross-modal feature fusion module and local scoring mechanism for noise suppression.
• Employed a feedback attention module in the decoder for informative feature selection.

Main Results:

• Achieved superior performance on GoPro, REBlur, and RwEvent datasets, exceeding REFID in PSNR and SSIM.
• Demonstrated significant improvements in pixel and structural restoration with minimal increase in parameters and computational cost.
• Ablation studies confirmed the positive contribution of each module.

Conclusions:

• The proposed network effectively addresses cross-modal mismatches in event-driven image deblurring.
• The method offers enhanced deblurring and generalization capabilities with efficient performance.
• The fusion strategy and attention mechanism significantly improve image reconstruction quality.