Real-world super-resolution with VLM-based degradation prior learning.

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Real-world image super-resolution (Real-SR) is challenged by complex, unknown degradations that hinder reconstruction accuracy.
• Accurate estimation of these real-world degradations is crucial for SR models to bridge the gap between synthetic training data and actual imaging conditions, yet this remains an open problem.

Purpose of the Study:

• To propose DePLSR, a method leveraging pre-trained vision-language models for effective degradation feature learning in Real-SR.
• To enhance the fidelity and quality of super-resolved images, particularly those affected by complex and ambiguous degradations.

Main Methods:

• DePLSR utilizes a Degradation Adaptor to predict degradation features from low-resolution (LR) images while preserving content.
• A Semantic-driven Degradation Pipeline and a mixed degradation dataset with captions were developed for training.
• A Cross-modal Fusion Module was introduced to integrate degradation information into downstream SR models.

Main Results:

• The DePLSR method achieved a notable improvement of 0.98 dB on Peak Signal-to-Noise Ratio (PSNR).
• Experiments demonstrated DePLSR's superior capability in extracting real image degradations and enhancing super-resolution performance on both synthetic and real-world datasets.
• Visualizations confirmed improved restoration for heavily degraded LR images and effective removal of complex degradations.

Conclusions:

• DePLSR effectively addresses the challenge of unknown degradations in Real-SR by learning degradation features through multimodal approaches.
• The proposed method significantly enhances image super-resolution performance, offering better restoration for complex and degraded real-world images.