MB-TaylorFormer V2: Improved Multi-Branch Linear Transformer Expanded by Taylor Formula for Image Restoration

Area of Science:

• Computer Vision
• Artificial Intelligence
• Deep Learning

Background:

• Transformer networks excel in image restoration due to global receptive fields.
• High computational complexity of Softmax-attention limits Transformer application in high-resolution image restoration.

Purpose of the Study:

• To develop an efficient Transformer variant for image restoration tasks.
• To address the quadratic computational complexity of Softmax-attention.

Main Methods:

• Proposed a Transformer variant using Taylor expansion to approximate Softmax-attention.
• Employed norm-preserving mapping to approximate the Taylor expansion remainder, achieving linear complexity.
• Introduced a multi-branch architecture with multi-scale patch embedding for enhanced feature processing.

Main Results:

• MB-TaylorFormer V2 demonstrates state-of-the-art performance across diverse image restoration benchmarks.
• Achieved superior results in image dehazing, deraining, desnowing, motion deblurring, and denoising.
• The model processes coarse-to-fine features and captures long-distance pixel interactions with minimal computational overhead.

Conclusions:

• MB-TaylorFormer V2 effectively reduces computational cost while maintaining high performance in image restoration.
• The novel architecture enhances feature processing and approximation accuracy.
• This model offers a computationally efficient solution for high-resolution image restoration challenges.