Attention Redundancy Reduction for Image Super-Resolution

Transformer-based models have demonstrated great promises in single image super-resolution (SISR), but our investigations find significant redundancy in terms of high mutual information across the attention maps, which is associated with reduced efficiency and degraded performance of SOTA models. To address the problem, here we propose a low redundancy attention network (LRAN). First, to mitigate the redundancy among heads, we introduce in the self-attention computation a multi-element mechanism, which allows for the incorporation of various types of self-attention, thus increasing inter-head diversity. Second, to address the redundancy among blocks, we propose the encapsulated architecture, in which enhanced local perception unit and gated multi-layer perceptron are designed to capture local information. Specifically, this architecture incorporates a single self-attention layer between several MLP layers. Subsequently, the proposed gated multi-layer perceptron significantly enhances the SR quality. Extensive experiments demonstrate that LRAN outperforms SOTA models in the task of lightweight SR, achieving a better trade-off between quality and speed. For instance, the proposed LRAN-light surpasses SwinIR-light by 0.32dB PSNR in $\times 4$ SR on Urban100, while running $\times 4$ faster.