Improving the Stability and Efficiency of Diffusion Models for Content Consistent Super-Resolution

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Pre-trained latent diffusion models (DMs) show promise for image super-resolution (SR).
• However, DM noise sampling introduces randomness and control issues in SR outputs.
• Existing acceleration methods struggle with generative capacity control.

Purpose of the Study:

• To develop a super-resolution method that enhances visual quality and ensures content consistency.
• To combine the strengths of diffusion models and generative adversarial networks for improved SR results.

Main Methods:

• A two-stage approach partitioning SR into structure reconstruction (DM) and detail enhancement (GAN).
• A non-uniform timestep sampling strategy using a single initial step followed by a few reverse steps for structure reconstruction.
• Fine-tuning a pre-trained variational auto-encoder decoder via adversarial GAN training for deterministic detail enhancement.

Main Results:

• The proposed Content Consistent Super-Resolution (CCSR) method significantly improves content consistency.
• CCSR maintains high perceptual quality even with a reduced number of diffusion steps (e.g., 1 or 2).
• The method allows flexible use of diffusion steps during inference without re-training.

Conclusions:

• CCSR effectively addresses the randomness and control issues in DM-based SR.
• The hybrid DM-GAN approach offers a robust solution for high-quality and consistent image super-resolution.
• The findings suggest a promising direction for controllable and efficient generative SR.