Diagnosing and Improving Vector-Quantization-Based Blind Image Restoration.

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Vector-Quantization (VQ) is commonly used in generative models for high-quality (HQ) priors in blind image restoration (BIR).
• Discrete VQ processes in BIR methods have limitations, including confined codebook capacity, error-prone index prediction on low-quality (LQ) images, and underutilization of LQ image features.
• These limitations hinder optimal performance in image restoration tasks.

Purpose of the Study:

• To address the limitations of discrete VQ in VQ-based BIR methods.
• To propose a novel approach using continuous feature representation and a Self-in-Cross-Attention (SinCA) module for enhanced image restoration.
• To improve restoration fidelity by better leveraging LQ image information.

Main Methods:

• Replaced the discrete Vector-Quantization (VQ) process with continuous feature representation.
• Introduced a Self-in-Cross-Attention (SinCA) module to augment the HQ codebook with input LQ image features.
• Implemented cross-attention between LQ features and the input-augmented codebook within a transformer architecture.

Main Results:

• VQ-based BIR methods enhanced with the proposed SinCA module and continuous representation showed improved quantitative and qualitative performance.
• The method demonstrated superior results in blind image super-resolution and blind face restoration tasks.
• Experiments confirmed the effectiveness of SinCA in leveraging LQ image features for better restoration fidelity.

Conclusions:

• Continuous feature representation and the SinCA module offer a significant improvement over traditional discrete VQ in BIR.
• The proposed SinCA module effectively enhances codebook representation by incorporating LQ image features, leading to superior restoration outcomes.
• The findings suggest a promising new direction for developing advanced blind image restoration techniques.