Learning Discrete Representations From Reference Images for Large Scale Factor Image Super-Resolution.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Image super-resolution (SR) aims to enhance low-resolution (LR) images to high-resolution (HR) versions.
• Deep neural networks have advanced SR, but large scale factors (e.g., >4x) remain challenging due to information loss.
• Traditional reference-based SR methods struggle with accurate texture matching in downscaled image or feature spaces.

Purpose of the Study:

• To address the challenges of reconstructing high-quality HR images from LR images at large scale factors.
• To exploit fine details from reference HR images for improved SR performance.
• To develop a novel method for learning and utilizing cross-scale discrete feature representations.

Main Methods:

• A convolutional neural network (CNN) auto-encoder module inspired by vector quantization (VQ) learns discrete image representations.
• Progressive learning of cross-scale discrete feature representations using paired LR and HR reference images.
• During inference, LR image features query discrete representations (value) retrieved by matching with coarser scale representations (key) to progressively recover the HR image.

Main Results:

• The proposed method achieves advanced performance compared to state-of-the-art image SR models.
• Demonstrates superior objective quality metrics for reconstructed HR images.
• Exhibits improved subjective visual quality in the super-resolved images.

Conclusions:

• The developed method effectively reconstructs HR images from LR inputs, particularly at large scale factors.
• The use of cross-scale discrete representations enhances detail recovery and overall image quality.
• The approach offers a promising solution for challenging image super-resolution tasks.