Learning an Adaptive Sparse Transformer for Efficient Image Restoration.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Transformer models excel in image restoration by capturing long-range dependencies.
• Existing efficient transformers struggle with redundant information and noisy interactions from irrelevant image regions.

Purpose of the Study:

• To develop an improved transformer model, Adaptive Sparse Transformer version 2 (AST-v2), for enhanced image restoration.
• To address the limitations of computational intensity and noisy interactions in current transformer architectures.

Main Methods:

• AST-v2 utilizes an Adaptive Sparse Self-Attention (ASSA) block with a dual-branch design to guide attention weights and reduce irrelevant token interactions.
• A Feature Refinement Feed-forward Network (FRFN) is employed to eliminate feature redundancy across channels.

Main Results:

• AST-v2 demonstrates competitive performance across six diverse image restoration tasks: rain streak removal, haze removal, shadow removal, snow removal, blur removal, and low-light enhancement.
• The proposed method effectively mitigates noisy interactions and feature redundancy, leading to clearer image restoration.

Conclusions:

• AST-v2 offers an efficient and effective solution for various image restoration challenges.
• The adaptive sparse attention and feature refinement mechanisms contribute to superior image quality and model efficiency.