Burst denoising transformer with multi-task optical flow estimation.

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Burst denoising aims to create clear images from rapid, noisy sequences.
• Frame misalignment due to camera or scene movement is a significant challenge in burst image capture.
• Existing methods struggle with effective frame alignment and noise reduction simultaneously.

Purpose of the Study:

• To introduce a novel network, the Burst Denoising Transformer (BDFormer), for effective burst denoising.
• To address the challenge of frame misalignment in burst image sequences.
• To improve the quality of denoised images while maintaining computational efficiency.

Main Methods:

• Developed a Transformer-based Multi-task Optical Flow Estimation (TMOFE) module for frame alignment, incorporating an auxiliary denoising task.
• Introduced a Transformer-based Feature Enrichment (TFE) module utilizing a Spatial and Channel-wise Transformer Block (SCTB).
• The SCTB combines FFT-based Spatial Transformer Blocks (FSTB) and Channel-wise Transformer Blocks (CTB) to leverage global spatial and channel information.

Main Results:

• BDFormer demonstrates superior performance compared to existing transformer-based denoising methods.
• The proposed TMOFE module effectively reduces noise impact during optical flow estimation.
• The SCTB effectively integrates inter- and intra-frame spatial and channel information for enhanced feature enrichment.

Conclusions:

• BDFormer offers a significant advancement in burst denoising technology.
• The novel architecture effectively handles frame misalignment and noise reduction.
• The method achieves state-of-the-art results with competitive computational complexity.