Adaptive Sparse Self-Attention for Efficient Image Super-Resolution and Beyond.

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Transformer-based self-attention mechanisms have advanced image super-resolution by modeling non-local features.
• Existing methods often use all token similarities, which can be inefficient and hinder high-quality reconstruction.
• Current self-attention approaches struggle with local feature exploration, impacting structural detail restoration.

Purpose of the Study:

• To develop an adaptive sparse self-attention method for improved image restoration.
• To enhance the utilization of relevant token information for feature aggregation.
• To better model both local and non-local features for superior structural detail restoration.

Main Methods:

• Developed a local spatial variant feature estimation method to generate query and key for self-attention, improving local information modeling.
• Introduced an adaptive sparse self-attention mechanism to select the most useful similarity values from the self-attention matrix.
• Integrated local and non-local feature modeling for enhanced image restoration.

Main Results:

• The proposed method effectively models both local and non-local image features.
• Achieved superior structural detail restoration compared to existing methods.
• Demonstrated favorable performance against state-of-the-art methods in accuracy and model complexity on benchmark datasets.

Conclusions:

• The adaptive sparse self-attention method offers a more effective approach to image restoration.
• This technique can serve as a valuable alternative to conventional self-attention mechanisms.
• The method shows significant potential for advancing high-quality image reconstruction.