Revealing the Dark Side of Non-Local Attention in Single Image Super-Resolution

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Single Image Super-Resolution (SISR) reconstructs high-resolution images from low-resolution inputs.
• Non-Local Attention (NLA) is a common technique using self-similar textures to improve SISR.
• Existing methods overlook NLA's potential to degrade texture quality.

Purpose of the Study:

• To challenge the assumption that NLA always improves SISR.
• To propose a sub-pixel level evaluation for NLA in SISR.
• To develop a strategy mitigating NLA-induced texture degradation.

Main Methods:

• Analyzing NLA's impact on texture fidelity at a sub-pixel level.
• Deriving an approximate reconstruction performance upper bound for NLA.
• Designing and integrating a Texture-Fidelity Strategy (TFS) into SISR models.
• Developing a Deep Texture-Fidelity Network (DTFN) incorporating TFS.

Main Results:

• NLA can severely distort textures in SISR, especially with degraded inputs.
• A sub-pixel evaluation reveals NLA's limitations.
• The proposed TFS effectively mitigates NLA-induced degradation.
• The DTFN achieves state-of-the-art SISR performance.

Conclusions:

• NLA is not universally beneficial for SISR and can degrade texture quality.
• A sub-pixel perspective and TFS are crucial for reliable NLA application in SISR.
• The DTFN offers a robust solution for high-fidelity SISR.