Restoring Vision in Adverse Weather Conditions With Patch-Based Denoising Diffusion Models.

Area of Science:

• Computer Vision
• Deep Learning
• Image Processing

Background:

• Adverse weather conditions significantly degrade image quality, impacting various computer vision applications.
• Recent advancements in deep neural networks, particularly vision transformers, have shown promise in image restoration.
• Conditional generative models have achieved state-of-the-art results, motivating new architectural designs.

Purpose of the Study:

• To present a novel patch-based image restoration algorithm using denoising diffusion probabilistic models.
• To enable size-agnostic image restoration through a guided denoising process across overlapping patches.
• To achieve state-of-the-art performance in both weather-specific and multi-weather image restoration.

Main Methods:

• Development of a patch-based diffusion modeling approach for image restoration.
• Implementation of a guided denoising process with smoothed noise estimates during inference.
• Empirical evaluation on benchmark datasets for desnowing, deraining, dehazing, and raindrop removal.

Main Results:

• The proposed model achieves state-of-the-art performance on various adverse weather image restoration tasks.
• Demonstrated effectiveness in both single-weather (desnowing, deraining, dehazing, raindrop removal) and multi-weather scenarios.
• Showcased strong generalization capabilities on real-world test images.

Conclusions:

• The patch-based diffusion modeling approach is effective for size-agnostic image restoration under adverse weather.
• The guided denoising process with smoothed noise estimates contributes to robust performance.
• The algorithm represents a significant advancement in handling diverse and challenging image degradation conditions.