All-in-One Transformer for Image Restoration Under Adverse Weather Degradations | JoVE Visualize

Area of Science:

Computer Vision
Image Restoration
Artificial Intelligence

Background:

Severe weather conditions cause multiple image degradations simultaneously in real-world scenarios.
Existing image restoration models struggle with accurately differentiating and addressing composite degradations due to reliance on scene descriptors.
This limitation leads to suboptimal performance in practical applications requiring robust image restoration.

Purpose of the Study:

To propose a novel Transformer-based restoration framework, AllRestorer, capable of adaptively handling multiple severe weather impairments.
To overcome the limitations of existing methods that focus on single or limited composite degradations.
To improve the accuracy and effectiveness of image restoration in complex, real-world weather conditions.

Main Methods:

Developed AllRestorer, a Transformer-based framework designed for simultaneous restoration of low-light, haze, rain, and snow impairments.
Introduced the All-in-One Transformer Block (AiOTB) enabling adaptive handling of multiple degradations within a single image.
Utilized Composite Scene Embedding (image and text embeddings) and adaptive degradation weighting within AiOTB for precise restoration control.

Main Results:

AllRestorer adaptively considers all weather impairments, avoiding misdirection from inaccurate scene descriptors.
The All-in-One Transformer Block (AiOTB) effectively handles multiple degradations simultaneously, outperforming single-degradation Transformers.
Achieved a significant 5.00 dB increase in Peak Signal-to-Noise Ratio (PSNR) on the CDD-11 dataset compared to baseline methods.

Conclusions:

AllRestorer provides a robust solution for severe weather image restoration by adaptively addressing multiple simultaneous degradations.
The proposed AiOTB and Composite Scene Embedding offer a novel approach to enhance restoration accuracy and control.
This framework demonstrates a significant advancement in handling complex image restoration tasks in challenging environmental conditions.