INformer: Inertial-Based Fusion Transformer for Camera Shake Deblurring.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Inertial measurement units (IMUs) capture device motion via gyroscopes and accelerometers.
• Conventional deblurring methods often neglect IMU data, while existing IMU-based approaches struggle to fully utilize this information, leading to noise.
• Challenges exist in effectively integrating sensor-derived motion data into image deblurring processes.

Purpose of the Study:

• To propose a novel multi-stage deblurring network, INformer, that effectively integrates inertial measurement unit (IMU) data.
• To address the limitations of existing deblurring methods in leveraging IMU information and mitigating sensor noise.
• To enhance image deblurring performance by fusing motion data with image features using advanced attention mechanisms.

Main Methods:

• Developed a multi-stage deblurring network, INformer, incorporating Transformer architecture.
• Designed an IMU-image Attention Fusion (IAF) block to merge motion information with blurry image features at the attention level.
• Introduced an Inertial-Guided Deformable Attention (IGDA) block to use motion features for adaptive receptive field adjustment and blur kernel refinement.

Main Results:

• The proposed INformer network demonstrated superior performance in image deblurring tasks.
• Extensive experiments on comprehensive benchmarks confirmed the method's effectiveness.
• The IAF and IGDA blocks successfully integrated inertial data, improving blur kernel estimation.

Conclusions:

• INformer effectively utilizes IMU data for advanced image deblurring.
• The proposed attention fusion and guided deformable attention blocks significantly enhance deblurring quality.
• The method shows favorable performance compared to state-of-the-art deblurring approaches.